Search

CN-122023127-A - Wavelet guided direction-alternating state space super-resolution reconstruction method, system, storage medium and device

CN122023127ACN 122023127 ACN122023127 ACN 122023127ACN-122023127-A

Abstract

A wavelet guided directional alternation state space super-resolution reconstruction method, a system, a storage medium and a device belong to the technical field of computer vision. The method aims to solve the technical problem that the prior art is difficult to consider the technical problems of long-range structural consistency and high-frequency detail fidelity under the light weight constraint. Firstly, extracting overcomplete shallow layer characteristics of a low-resolution image to be processed as initial input characteristics, obtaining deep layer characteristics through characteristic modulation for a plurality of times, and carrying out up-sampling after combining the shallow layer characteristics and the deep layer characteristics to obtain a high-resolution image. And parallel processing of discrete wavelet decomposition and image space line-row alternate scanning modeling is introduced in the characteristic modulation process, and both long-range structural consistency and high-frequency detail fidelity are considered. The method is mainly used for super-resolution reconstruction of the image with low resolution.

Inventors

  • YANG HONG
  • CHI JING
  • YANG XIANQIANG

Assignees

  • 哈尔滨工业大学

Dates

Publication Date
20260512
Application Date
20260212

Claims (10)

  1. 1. The wavelet guided direction-alternating state space super-resolution reconstruction method is characterized by comprising the following steps of: s1, extracting overcomplete shallow layer characteristics of a low-resolution image to be processed; Step S2, taking overcomplete shallow layer characteristics as initial input, sequentially executing G times of characteristic modulation processing, sequentially executing B times of sub-processing each time of characteristic modulation processing by taking the current input characteristics as starting points, and connecting the output characteristics of the B times of sub-processing with the input characteristics of the current characteristic modulation processing through residual errors after convolution; The 1 st sub-process takes the input characteristic of the characteristic modulation process as input, and the subsequent sub-processes take the output result of the previous sub-process as input, wherein each sub-process comprises: (1) Dividing the input features of the sub-process into left features and right features; (2) Performing wavelet guided feature transformation processing on the left feature, wherein the processing comprises performing convolution operation on the left feature to generate a guide graph, performing discrete wavelet processing operation on the guide graph to obtain two first-stage sub-features, performing multi-stage feature transformation processing on the two first-stage sub-features to obtain guide weights, and performing element-by-element weighted correction on the left feature by using the guide weights to obtain first features; The method comprises the steps of carrying out normalization processing, linear transformation processing, convolution processing, activation processing, line or column space scanning modeling processing and normalization processing on right features in sequence to generate first intermediate features, and carrying out linear transformation processing and activation processing on the right features in sequence to obtain second intermediate features; (3) The method comprises the steps of generating a fusion feature based on the first feature and the second feature, connecting the fusion feature with the input feature of the sub-process through residual errors to obtain a first residual error feature, grouping the first residual error feature, inputting a plurality of volumes of integral branches for parallel processing, splicing the output features of all convolution branches, and connecting the output features with the first residual error feature through residual errors to obtain a second residual error feature; And step S3, obtaining a trunk feature based on the overcomplete shallow layer feature and the deep layer reconstruction feature, and performing up-sampling operation on the trunk feature to obtain a high-resolution image.
  2. 2. The method for reconstructing the spatial super-resolution of the wavelet guided direction alternation state of the invention as set forth in claim 1, wherein the extracting process of the overcomplete shallow features comprises copying the channel dimension of the low resolution image for a preset number of times, splicing the image to obtain overcomplete input data, and mapping the overcomplete input data into the overcomplete shallow features with a preset number of channels through convolution projection.
  3. 3. The method according to claim 2, wherein in step S2, in the performing wavelet guided feature transformation processing on the left feature, the two first-stage sub-features include a low-frequency sub-band and a high-frequency sub-band; The multi-order feature transformation processing process comprises the steps of constructing a low-frequency energy diagram and a high-frequency energy diagram based on the low-frequency sub-band and the high-frequency sub-band and taking an overcomplete shallow feature size as a reference, constructing a residual energy diagram based on the low-frequency energy diagram and the high-frequency energy diagram, performing preset amplitude clipping and normalization operation on the residual energy diagram to obtain an initial weight diagram, and performing edge-preserving smoothing processing on the initial weight diagram to obtain guide weights.
  4. 4. A method of reconstructing a spatial super-resolution of a wavelet guided direction-alternating state according to claim 3, wherein said edge preserving smoothing is implemented by using a guided filtering method, said guided graph is used as a guiding signal, and said initial weight graph is used as an input signal.
  5. 5. The method for reconstructing a wavelet guided spatially super-resolution in an alternating state according to claim 1, wherein said line or column space scan modeling process comprises any one of the following scan rules; In the even-numbered feature modulation processing, the corresponding sub-processing adopts column-direction state space scanning modeling; In the odd-number feature modulation processing, the corresponding sub-processing adopts column-direction state space scanning modeling; in each feature modulation process, the odd sub-process adopts line-direction state space scanning modeling, and the even sub-process adopts column-direction state space scanning modeling; and in each feature modulation process, the odd sub-process adopts column-wise state space scanning modeling, and the even sub-process adopts line-wise state space scanning modeling.
  6. 6. The method for reconstructing the spatial super-resolution of the direction-alternating state guided by the wavelet of claim 5, wherein the specific process of generating the fusion feature based on the first feature and the second feature comprises the steps of splicing the first feature and the second feature along the channel dimension and then carrying out channel shuffling to obtain the fusion feature.
  7. 7. The method for reconstructing the spatial super-resolution of the direction-alternating state guided by the wavelet of claim 1, wherein step S2 is performed by grouping the first residual features and then inputting the first residual features into multiple convolution branches for parallel processing, and specifically comprises: after carrying out layer normalization on the first residual characteristic, dividing the first residual characteristic into three groups of characteristics according to channels, and respectively passing the three groups of characteristics Deep convolution branch, Deep convolution branches And (3) carrying out deep convolution branch processing, wherein k takes a value of 5 or 7 or 11.
  8. 8. The wavelet guided directional alternation state space super-resolution reconstruction system is characterized by comprising an overcomplete shallow representation module, a residual dual-path characteristic modulation group module and an up-sampling reconstruction module; The complete shallow characterization module is used for extracting overcomplete shallow characteristics of the low-resolution image to be processed; the residual dual-path characteristic modulation group module comprises G cascaded residual dual-path characteristic modulation groups, wherein the residual dual-path characteristic modulation group comprises B dual-branch cooperative units and a group tail convolution aggregation unit; The residual dual-path characteristic modulation group module takes overcomplete shallow characteristics as initial input, sequentially processes G cascaded residual dual-path characteristic modulation groups, sequentially processes B dual-branch cooperative units by taking the current input characteristics as starting points, and performs convolution operation by a tail convolution aggregation unit; the double-branch cooperative unit comprises a wavelet guide enhancement branch, a direction alternating state space branch, a characteristic fusion and interaction branch, an anisotropic refinement branch and a feedforward neural network branch; The wavelet guide enhancement branch is used for executing wavelet guide feature transformation processing on the current input feature, and specifically, generating a guide graph based on the left feature obtained by dividing the current input feature, executing discrete wavelet processing operation on the guide graph to obtain two first-stage sub-features; The direction alternating state space branch is used for parallelly executing a first sub-branch process and a second sub-branch process on the left feature obtained based on the current input feature division, wherein the first sub-branch process comprises the steps of sequentially carrying out normalization process, linear transformation process, convolution process, activation process, line or column space scanning modeling process and normalization process on the right feature to generate a first intermediate feature; The feature fusion and interaction branch is used for generating fusion features based on the first features and the second features; The anisotropic refinement branch is used for connecting the fusion feature and the overcomplete shallow layer feature through residual errors to obtain a first residual error feature, grouping the first residual error feature, inputting the first residual error feature into a multi-volume integration branch for parallel processing, splicing the output features of all convolution branches, and connecting the output features with the first residual error feature through residual errors to obtain a second residual error feature; the feedforward neural network branch is used for obtaining output characteristics based on the second residual characteristics and the overcomplete shallow layer characteristics; the up-sampling reconstruction module is used for performing up-sampling operation on the deep reconstruction features to obtain a high-resolution image.
  9. 9. A computer-readable storage medium comprising instructions that, when run on a computing device, cause the computing device to perform the method of any of claims 1-7.
  10. 10. A computing device, the computing device comprising a processor and a memory: the memory is used for storing a computer program; The processor is configured to perform the method of any one of claims 1 to 7 according to the computer program.

Description

Wavelet guided direction-alternating state space super-resolution reconstruction method, system, storage medium and device Technical Field The invention relates to the technical field of computer vision, in particular to an image super-resolution reconstruction method, an image super-resolution reconstruction system, a storage medium and image super-resolution reconstruction equipment. Background The image super-resolution reconstruction aims to recover an image result with higher resolution by low-resolution observation so as to improve detail intelligibility and structural definition, and has important value in applications such as video monitoring evidence collection, satellite and aerial remote sensing, medical image auxiliary diagnosis, mobile end imaging enhancement and the like. With the development of deep learning, the super-resolution algorithm based on the neural network is continuously improved in the aspects of reconstruction precision and subjective impression, wherein the convolutional neural network can better capture local modes such as textures, edges and the like by virtue of the characteristics of local connection and parameter sharing, and a stable reconstruction effect is obtained under a multi-scale feature fusion frame. In recent years, a state space model is introduced into an image restoration task due to the advantage of linear complexity, but the existing method scans along a fixed direction to cause insufficient directional expression, and the cooperation of a state space branch and a convolution branch lacks the pertinence compensation of local texture prior, and meanwhile, lacks an explicit high-frequency guiding mechanism, so that the long-range structural consistency and high-frequency detail fidelity are difficult to be considered under the light weight constraint. Disclosure of Invention The application provides a wavelet guided direction alternating state space super-resolution reconstruction method and a wavelet guided direction alternating state space super-resolution reconstruction system, which aim to solve the technical problem that the existing image super-resolution reconstruction method is difficult to consider long-range structural consistency and high-frequency detail fidelity under the light weight constraint. Furthermore, the application also provides a corresponding computer readable storage medium and equipment. The first aspect of the present invention provides a wavelet guided direction-alternating state spatial super-resolution reconstruction, comprising: s1, extracting overcomplete shallow layer characteristics of a low-resolution image to be processed; Step S2, taking overcomplete shallow layer characteristics as initial input, sequentially executing G times of characteristic modulation processing, sequentially executing B times of sub-processing each time of characteristic modulation processing by taking the current input characteristics as starting points, and connecting the output characteristics of the B times of sub-processing with the input characteristics of the current characteristic modulation processing through residual errors after convolution; The 1 st sub-process takes the input characteristic of the characteristic modulation process as input, and the subsequent sub-processes take the output result of the previous sub-process as input, wherein each sub-process comprises: (1) Dividing the input features of the sub-process into left features and right features; (2) Performing wavelet guided feature transformation processing on the left feature, wherein the processing comprises performing convolution operation on the left feature to generate a guide graph, performing discrete wavelet processing operation on the guide graph to obtain two first-stage sub-features, performing multi-stage feature transformation processing on the two first-stage sub-features to obtain guide weights, and performing element-by-element weighted correction on the left feature by using the guide weights to obtain first features; The method comprises the steps of carrying out normalization processing, linear transformation processing, convolution processing, activation processing, line or column space scanning modeling processing and normalization processing on right features in sequence to generate first intermediate features, and carrying out linear transformation processing and activation processing on the right features in sequence to obtain second intermediate features; (3) The method comprises the steps of generating a fusion feature based on the first feature and the second feature, connecting the fusion feature with the input feature of the sub-process through residual errors to obtain a first residual error feature, grouping the first residual error feature, inputting a plurality of volumes of integral branches for parallel processing, splicing the output features of all convolution branches, and connecting the output features with the first residual error featu