CN-122024042-A - Method for translating high-resolution synthetic aperture radar image into multispectral image

CN122024042ACN 122024042 ACN122024042 ACN 122024042ACN-122024042-A

Abstract

The invention discloses a method for translating a high-resolution synthetic aperture radar image into a multispectral image, which comprises the steps of constructing a generator network and a recognizer network, wherein the generator network comprises a TV-BM3D module and a generator module, the recognizer network comprises a thermodynamic constraint module and a recognizer module, acquiring a training data set, training the generator network by utilizing the training data set and the recognizer network to acquire a trained recognizer network, and inputting an original high-resolution SAR image to be processed into the trained recognizer network to acquire a corresponding multispectral image. The method utilizes the variable block matched filtering technology to inhibit the high-resolution SAR speckle noise and simultaneously retain edge and texture details, and takes the thermodynamic first law as the prior to carry out conservation constraint on energy flow in the generation process, so that the cross-modal mapping can be stabilized and the spectrum fidelity and channel coordination of the multispectral image can be improved.

Inventors

Han Qingcen
ZHOU YING
JI XINCHUN
YANG HONG
YANG CHUNXIAO
LIU YANG

Assignees

西北工业大学

Dates

Publication Date: 20260512
Application Date: 20260113

Claims (9)

1. A method for translating a high resolution synthetic aperture radar image into a multispectral image, comprising: S1, constructing a generator network and a recognizer network, wherein the generator network comprises a TV-BM3D module and a generator module, the TV-BM3D module is used for carrying out coupling denoising processing on an input SAR image to obtain a denoised SAR image, and the generator module is used for obtaining a corresponding prediction multispectral image by utilizing the denoised SAR image; the identifier network comprises a thermodynamic constraint module and an identifier module, wherein the thermodynamic constraint module is used for carrying out thermodynamic constraint processing on a real multispectral image and a predicted multispectral image in the training process of the generator network to obtain a constrained real multispectral image and a predicted multispectral image, and the identifier module is used for obtaining an identification result by utilizing the constrained real multispectral image and the constrained predicted multispectral image and updating parameters of the generator network; s2, acquiring a training data set, and training the generator network by utilizing the training data set and the identifier network to obtain a trained identifier network; s3, inputting the original SAR image to be processed into a trained identifier network to obtain a corresponding multispectral image.
2. The method for translating a high resolution synthetic aperture radar image into a multispectral image according to claim 1, wherein the TV-BM3D module specifically comprises a variational regularization sub-module, a three-dimensional block-matched filtering sub-module, and a laplace pyramid sub-module, wherein, The variation regularization submodule is used for primarily denoising the input SAR image by restraining the image gradient norm minimization to obtain a primarily denoised SAR image; the three-dimensional block matching filtering sub-module is used for filtering and denoising the input SAR image by utilizing the statistical correlation of a large number of similar blocks in the SAR image to obtain a filtered and denoised SAR image; the Laplacian pyramid sub-module is used for carrying out image fusion on the primarily denoised SAR image and the filtered denoised SAR image to obtain a final denoised SAR image.
3. The method for translating a high resolution synthetic aperture radar image into a multispectral image according to claim 2, wherein the processing expression of the variational regularization sub-module is: , Wherein, the Represents an input SAR image containing noise, Representing the SAR image preliminarily denoised by the variational regularization submodule, Representing the regularization of the total variation, parameters Is a regularization coefficient.
4. The method of translating a high resolution synthetic aperture radar image to a multispectral image according to claim 2, wherein the three-dimensional block matched filtering submodule includes a first SSIM block matching unit, an adaptive threshold transform unit, a first inverse transform and aggregation unit, a second SSIM block matching unit, a collaborative filtering unit, and a second inverse transform and aggregation unit, wherein, The first SSIM block matching unit is used for dividing an input SAR image into a plurality of image blocks with the same size, and classifying the image blocks by using an SSIM method to form a first three-dimensional array; The self-adaptive threshold conversion unit is used for carrying out separable conversion on the first three-dimensional array in two space dimensions and one grouping dimension respectively to obtain three-dimensional conversion coefficients, and carrying out threshold filtering on the three-dimensional conversion coefficients by utilizing a set self-adaptive threshold to obtain three-dimensional conversion coefficients after threshold filtering; The first inverse transformation and aggregation unit is used for carrying out inverse separable transformation on the three-dimensional transformation coefficient subjected to threshold filtering to obtain a denoised block group, and each image block in the denoised block group is overlapped back to a corresponding position of an original SAR image to obtain an output image of the first inverse transformation and aggregation unit; The second SSIM block matching unit is used for dividing the output image of the first inverse transformation and aggregation unit into a plurality of image blocks with the same size, classifying the current image block by using an SSIM method, and forming a second three-dimensional array; The coordination and simultaneous filtering unit is used for carrying out separable transformation on the second three-dimensional array in two space dimensions and one grouping dimension respectively to obtain a transformation domain coefficient, and carrying out wiener filtering on the transformation domain coefficient to obtain a transformation domain coefficient after wiener filtering; And the second inverse transformation and aggregation unit is used for carrying out inverse separable transformation on the transform domain coefficient after the wiener filtering to obtain a denoised block group, and overlapping each image block in the current denoised block group back to the corresponding position of the original SAR image through weighted average to obtain an output image of the second inverse transformation and aggregation unit.
5. The method for translating a high resolution synthetic aperture radar image into a multispectral image according to claim 4, wherein the first SSIM block matching unit is specifically configured to: Dividing an input SAR image into N multiplied by N image blocks with the same size, taking each image block as a reference block y, and searching K image blocks with the maximum SSIM value of the reference block y in the SAR image as similar blocks of the current reference block y, wherein the SSIM values of the two image blocks are expressed as follows: , , , , Wherein, the SSIM values representing reference block y and image block x, A luminance similarity term representing the reference block y and the image block x, A contrast similarity term representing the reference block y and the image block x, A structural similarity term representing the reference block y and the image block x, Representing the pixel mean of the image block x, Represents and references the pixel mean of the block y, Representing the standard deviation of the pixels of the image block x, Representing the standard deviation of the pixels of the reference block y, Representing the pixel covariance of image block x and reference block y, , And Is constant.
6. The method for translating a high resolution synthetic aperture radar image into a multispectral image according to claim 1, wherein the thermodynamic constraint module is specifically configured to: Respectively calculating the gradient in the horizontal direction and the gradient in the vertical direction of an input multispectral image to obtain the horizontal gradient characteristic and the vertical gradient characteristic of the multispectral image, and carrying out characteristic fusion on the horizontal gradient characteristic and the vertical gradient characteristic of the multispectral image to obtain the fusion gradient characteristic of the multispectral image; collecting and polymerizing the fusion gradient characteristics by using a pyramid model to obtain the polymerized characteristics; And processing the aggregation characteristics of the real multispectral image and the aggregation characteristics of the predicted multispectral image by utilizing modulation gating to respectively obtain the constrained image of the real multispectral image and the constrained image of the predicted multispectral image.
7. The method of claim 6, wherein the identifier module is configured to receive the constrained image of the real multispectral image and the constrained image of the predicted multispectral image, output a recognition result, and update parameters of the generator network according to the recognition result.
8. The method of translating a high resolution synthetic aperture radar image to a multispectral image according to claim 6, wherein processing the aggregate features of the true multispectral image and the aggregate features of the predicted multispectral image using modulation gating to obtain the constrained image of the true multispectral image and the constrained image of the predicted multispectral image, respectively, comprises: Generating modulation gating for suppressing a region of an image that differs greatly from the original image: Wherein, the As a function of the Sigmoid, Representing the aggregate characteristics of a real multispectral image or the aggregate characteristics of a predicted multispectral image, Representing the gating control intensity, and modulating the constraint intensity of a sensitive area of the structure; Adding the constrained aggregation features to the input image of the thermodynamic constraint module by adopting residual type reprojection to obtain a constrained image: , Wherein, the For the multiplication on an element-by-element basis, Is a true multispectral image or a predicted multispectral image constrained image.
9. The method of translating a high resolution synthetic aperture radar image into a multispectral image according to claim 6, wherein S2 comprises: Acquiring a preset amount of SAR images and multispectral images corresponding to each SAR image to form a pairing data set, and preprocessing each image in the pairing data set to form a training data set; Inputting SAR images in the training data set into the generator network, and outputting predicted multispectral images; The predicted multispectral image and the real multispectral image corresponding to the SAR image are input into the identifier network together, so that an identification result is obtained, and parameters of the generator network are updated according to the identification result; And carrying out iterative training on the generator network by utilizing the training data set, and obtaining the trained generator network after the preset training requirement is met.

Description

Method for translating high-resolution synthetic aperture radar image into multispectral image Technical Field The invention belongs to the technical field of artificial intelligence, and particularly relates to a method for translating a high-resolution synthetic aperture radar image into a multispectral image. Background The existing high-altitude remote sensing reconnaissance image comprises Synthetic Aperture Radar (SAR) imaging and multispectral images, and has important value in tasks such as forest resource evaluation, agricultural monitoring, disaster response, earth surface coverage monitoring and the like. The SAR adopts a microwave active transmitting and echo receiving mechanism, has all-weather observation capability in all weather and can stably acquire images under complex atmospheric conditions. However, SAR images have inherent speckle noise and high frequency texture fluctuations, which are difficult to interpret directly. The multispectral imaging can collect a plurality of spectral bands including visible light and near infrared light at the same time, the spectral resolution of the multispectral image is higher, interpretation is visual, but high-quality images are difficult to obtain under imaging conditions of a sensor, such as night, severe weather conditions of haze, cloudiness and the like. Therefore, how to obtain the remote sensing image which is convenient to interpret in bad weather has become a research important point and difficulty in the remote sensing detection work. At present, an inter-domain image translation method is mainly adopted in image conversion, but at present, the main stream method is translation among natural images, and high-resolution remote sensing images are not subjected to targeted processing, wherein speckle noise and high-frequency disturbance of high-resolution SAR are prominent, artifact and detail distortion are often induced to be generated, and the image quality and stability are low. And correlation and spectrum consistency constraint in the multispectral image is easy to generate false color and channel inconsistency, meanwhile, the disclosed data is inconsistent in multisource, the resolution is low, and pixel level pairing is difficult. Disclosure of Invention In order to solve the problems in the prior art, the invention provides a method for translating a high-resolution synthetic aperture radar image into a multispectral image. The technical problems to be solved by the invention are realized by the following technical scheme: The invention provides a method for translating a high-resolution synthetic aperture radar image into a multispectral image, which comprises the following steps: S1, constructing a generator network and a recognizer network, wherein the generator network comprises a TV-BM3D module and a generator module, the TV-BM3D module is used for carrying out coupling denoising processing on an input SAR image to obtain a denoised SAR image, and the generator module is used for obtaining a corresponding prediction multispectral image by utilizing the denoised SAR image; the identifier network comprises a thermodynamic constraint module and an identifier module, wherein the thermodynamic constraint module is used for carrying out thermodynamic constraint processing on a real multispectral image and a predicted multispectral image in the training process of the generator network to obtain a constrained real multispectral image and a predicted multispectral image, and the identifier module is used for obtaining an identification result by utilizing the constrained real multispectral image and the constrained predicted multispectral image and updating parameters of the generator network; s2, acquiring a training data set, and training the generator network by utilizing the training data set and the identifier network to obtain a trained identifier network; s3, inputting the original SAR image to be processed into a trained identifier network to obtain a corresponding multispectral image. In one embodiment of the invention, the TV-BM3D module specifically includes a variational regularization sub-module, a three-dimensional block-matching filtering sub-module, and a laplacian pyramid sub-module, where, The variation regularization submodule is used for primarily denoising the input SAR image by restraining the image gradient norm minimization to obtain a primarily denoised SAR image; the three-dimensional block matching filtering sub-module is used for filtering and denoising the input SAR image by utilizing the statistical correlation of a large number of similar blocks in the SAR image to obtain a filtered and denoised SAR image; the Laplacian pyramid sub-module is used for carrying out image fusion on the primarily denoised SAR image and the filtered denoised SAR image to obtain a final denoised SAR image. In one embodiment of the present invention, the processing expression of the variation regularization sub-module is: , Wherei