CN-121147019-B - Image enhancement method based on spatial spectrum aggregation and interaction

CN121147019BCN 121147019 BCN121147019 BCN 121147019BCN-121147019-B

Abstract

The invention provides an image enhancement method based on spatial spectrum aggregation and interaction, which comprises the steps of obtaining a first hyperspectral image to be processed, and processing the first hyperspectral image by using a trained two-dimensional aggregation interaction network to obtain a second hyperspectral image, wherein the resolution of the second hyperspectral image is higher than that of the first hyperspectral image, and the two-dimensional aggregation interaction network comprises a group feature extraction module, a global feature extraction module, bicubic interpolation up-sampling and a high-resolution image reconstruction module. The group feature extraction module and the global feature extraction module systematically utilize the correlation between the dimension and the dimension through multi-scale aggregation in the dimension and interaction between the dimension, and finally, the high-resolution image reconstruction unit is utilized to reconstruct the image, so that excellent hyperspectral image enhancement task performance is realized, and excellent performance is realized under various super-resolution tasks.

Inventors

WENG SHIZHUANG
ZHENG LING
GE FEI
WANG XUFEI
WU QI
ZHANG QICHENG
ZHANG CHEN
CHEN JIALING
ZHU NINA
FU JIAXIN

Assignees

安徽大学

Dates

Publication Date: 20260505
Application Date: 20250822

Claims (7)

1. An image enhancement method based on spatial spectrum aggregation and interaction, which is characterized by comprising the following steps: Acquiring a first hyperspectral image to be processed; Processing the first hyperspectral image by using a trained two-dimensional aggregation interaction network to obtain a second hyperspectral image, wherein the resolution of the second hyperspectral image is higher than that of the first hyperspectral image; the expression of the two-dimensional aggregation interaction network is as follows: I SR =H RU (H GU (H SU (I LR )),I LR ↑), Wherein, I LR is the first hyperspectral image, I SR is the second hyperspectral image, H SU and H GU are a group feature extraction module and a global feature extraction module respectively to perform multi-scale aggregation in dimension and inter-dimension interaction, I LR ∈ represents performing bicubic interpolation up-sampling on the first hyperspectral image, H RU is a high-resolution image reconstruction module; The group feature extraction module consists of G double extraction interaction units and a convolution layer; the group feature extraction module processes the first hyperspectral image as follows: dividing the first hyperspectral image into G groups along a channel dimension; extracting the characteristics of G groups by using G double extraction interaction units respectively; splicing the features extracted by all G double-extraction interaction units; Mapping the spliced features to higher dimensionality by using a convolution layer to obtain output features of the group of feature extraction modules; The expression of the global feature extraction module is as follows: X GU-out =H DITB (X GU-in )+X GU-in , Wherein X GU-in and X GU-out are respectively input features and output features of the global feature extraction module, and H DITB is a double-interaction transducer unit; the expression of the high-resolution image reconstruction module is as follows: I SR =Conv(Upsampling(X GU-out )+Conv(I LR ↑)), Wherein, X GU-out is the output feature of the global feature extraction module, upsampling is Upsampling, conv is convolution.
2. The spatial spectrum aggregation and interaction based image enhancement method according to claim 1, wherein the expression of the dual extraction interaction unit is as follows: F spa =SPM(X DEIB-in )·SpeIM(SWM(X DEIB-in )), F spe =SWM(X DEIB-in )·SpaIM(SPM(X DEIB-in )), X DEIB-out =Conv(F spa +F spe ), Wherein X DEIB-in and X DEIB-out are respectively input features and output features of the dual extraction interactive unit, SPM is a spatial pyramid block, SWM is a spectral weight block, speIM is a spectral interactive block, spaIM is a spatial interactive block, and F spa and F spe are intermediate features.
3. The spatial spectrum aggregation and interaction based image enhancement method according to claim 2, wherein the expression of the spatial pyramid block is as follows: X 1 =ReLU(DConv(X SPM-in )), X 2 =ReLU(DConv(Downsampling(X 1 ))), X 3 =ReLU(Conv(Downsampling(X 2 ))), X SPM-out =Conv(Upsampling(Conv(Upsampling(X 3 )+X 2 ))+X 1 ), Wherein X SPM-in and X SPM-out are respectively the input features and the output features of the spatial pyramid block, DConv is depth separable convolution, reLU is an activation function, conv is convolution, downsampling is downsampling, upsampling, and X 1 ~X 3 is an intermediate feature.
4. The image enhancement method based on spatial spectral aggregation and interaction according to claim 2, wherein the spectral weighting block processes the input features X SWM-in as follows: Performing convolution operation on the input feature X SWM-in with the size of C multiplied by H multiplied by W to obtain a feature with the unchanged size, and performing remolding operation to obtain a first feature with the size of C multiplied by HW; Performing convolution operation on the input feature X SWM-in , compressing the channel number of the input feature X SWM-in to 1, and then performing remolding operation to obtain a second feature with the size HW multiplied by 1; performing matrix multiplication on the first feature and the second feature, and performing reshaping operation again to obtain a third feature X 4 with a size of Cx1x1; The input feature X SWM-in and the third feature X 4 are processed as follows: X SWM-out =(ReLU(X SWM-in )·ReLU(Sigmoid(X 4 )), wherein X SWM-out is the output characteristic of the spectrum weight block.
5. The spatial spectrum aggregation and interaction based image enhancement method according to claim 2, wherein the expression of the spatial interaction block is as follows: X 5 =GeLU(Conv(X SpaIM-in )), X 6 =AxialShift(X 5 )+X 5 , X SpaIM-out =Sigmoid(Conv(X 6 )), Where X SpaIM-in and X SpaIM-out are the input and output features of the spatial interaction block, respectively, geLU is the activation function, axialShift is the axial displacement, and X 5 and X 6 are the intermediate features.
6. The spatial spectral aggregation and interaction based image enhancement method according to claim 2, wherein the expression of the spectral interaction block is as follows: X 7 =AvgPool(X SpeIM-in ), X 8 =GeLU(Conv(X 7 ))+ChannelMix(GeLU(GConv(X 7 ))), X SpeIM-out =Sigmoid(Conv(X 8 )), Where X SpeIM-in and X SpeIM-out are the input and output features of the spectral interaction block, respectively, avgPool is average pooling, channelMix is channel shuffling, GConv is packet convolution, and X 7 and X 8 are intermediate features.
7. The method for spatial spectrum aggregation and interaction based image enhancement according to claim 1, wherein the expression of the double interaction transducer unit is as follows: Z spa =MSSA(LN(X DITB-in ))·SpeIM(LN(X DITB-in )), Z spe =WSSA(LN(X DITB-in ))·SpaIM(LN(X DITB-in )), Z’=(Z spa +Z spe )+X DITB-in , X DITB-out =MLP(LN(Z’))+Z’, wherein X DITB-in and X DITB-out are respectively the input characteristics and the output characteristics of the double-interaction transducer unit, MSSA is a mask space self-attention block, WSSA is a window spectrum self-attention block, speIM is a spectrum interaction block, spaIM is a space interaction block, LN is layer normalization, and MLP is a multi-layer perceptron.

Description

Image enhancement method based on spatial spectrum aggregation and interaction Technical Field The invention relates to the technical field of image enhancement, in particular to an image enhancement method based on spatial spectrum aggregation and interaction. Background The hyperspectral image (HYPERSPECTRAL IMAGE, HSI) has tens to hundreds of wave bands, provides rich spectral information for different substances and targets, and is widely applied to the fields of medical diagnosis, mineral exploration, ground object detection and the like. However, due to limitations of imaging equipment, the spatial resolution of HSI tends to be low, resulting in loss of detail information, including small spatial textures and details. The hyperspectral image Super-Resolution (HYPERSPECTRAL IMAGE Super-Resolution, HISR) means that a hyperspectral image (Low-Resolution HYPERSPECTRAL IMAGE, LRHSI) with Low spatial Resolution is generated into a hyperspectral image (High-Resolution HYPERSPECTRAL IMAGE, HRHSI). Therefore, HISR technology has important research significance in various fields. Recently, many HISR methods have been proposed, however, existing such methods ignore spatial and spectral intra-and inter-dimensional correlations, resulting in poor quality of super resolution. How to effectively and comprehensively utilize the similarity and complementarity of spatial and spectral information remains a challenging problem. Disclosure of Invention In view of the defects in the prior art, the invention provides an image enhancement method based on spatial spectrum aggregation and interaction, which aims to solve the technical problem that the super-resolution quality of a hyperspectral image is poor in the prior art. In order to achieve the above and other related objects, the present invention provides an image enhancement method based on spatial spectrum aggregation and interaction, comprising the steps of obtaining a first hyperspectral image to be processed, processing the first hyperspectral image by using a trained two-dimensional aggregation interaction network to obtain a second hyperspectral image, wherein the resolution of the second hyperspectral image is higher than that of the first hyperspectral image, and the expression of the two-dimensional aggregation interaction network is as follows: ISR＝HRU(HGU(HSU(ILR)),ILR↑), Wherein I LR is the first hyperspectral image, I SR is the second hyperspectral image, H SU and H GU are a group feature extraction module and a global feature extraction module respectively to perform multi-scale aggregation in dimensions and inter-dimension interaction, I LR ∈ represents performing bicubic interpolation up-sampling on the first hyperspectral image, and H RU is a high-resolution image reconstruction module. In an embodiment of the invention, the group feature extraction module is composed of G double-extraction interaction units and a convolution layer, and the group feature extraction module processes the first hyperspectral image according to the following steps that the first hyperspectral image is divided into G groups along a channel dimension, the G double-extraction interaction units are used for respectively extracting the features of the G groups, the features extracted by all the G double-extraction interaction units are spliced, and the spliced features are mapped to a higher dimension by the convolution layer to obtain the output features of the group feature extraction module. In an embodiment of the present invention, the expression of the dual extraction interaction unit is as follows: Fspa＝SPM(XDEIB-in)·SpeIM(SWM(XDEIB-in)), Fspe＝SWM(XDEIB-in)·SpaIM(SPM(XDEIB-in)), XDEIB-out＝Conv(Fspa+Fspe), Wherein X DEIB-in and X DEIB-out are respectively input features and output features of the dual extraction interactive unit, SPM is a spatial pyramid block, SWM is a spectral weight block, speIM is a spectral interactive block, spaIM is a spatial interactive block, and F spa and F spe are intermediate features. In an embodiment of the present invention, the expression of the spatial pyramid block is as follows: X1＝ReLU(DConv(XSPM-in)), X2＝ReLU(DConv(Downsampling(X1))), X3＝ReLU(Conv(Downsampling(X2))), XSPM-out＝Conv(Upsampling(Conv(Upsampling(X3)+X2))+X1), Wherein X SPM-in and X SPM-out are respectively the input features and the output features of the spatial pyramid block, DConv is depth separable convolution, reLU is an activation function, conv is convolution, downsampling is downsampling, upsampling, and X 1～X3 is an intermediate feature. In an embodiment of the present invention, the spectral weighting block processes the input feature X SWM-in by performing a convolution operation on the input feature X SWM-in having a size of c×h×w to obtain a feature having a constant size, and then performing a remodeling operation to obtain a first feature having a size of c×hw, performing a convolution operation on the input feature X SWM-in to compress the number of channel