CN-115601659-B - Remote sensing image missing information reconstruction method and device under radiation decoupling

CN115601659BCN 115601659 BCN115601659 BCN 115601659BCN-115601659-B

Abstract

The invention discloses a method and a device for reconstructing missing information of a remote sensing image under radiation decoupling, which can effectively reconstruct missing information caused by cloud and cloud shadow shielding and sensor faults in the remote sensing image. Firstly, decoupling radiation information recorded by a remote sensing image into an intrinsic content radiation component C related to earth surface knowledge and imaging condition radiation components A and B related to an imaging environment through a self-coding network, then constructing a radiation decoupling model according to a radiation correction model, designing a remote sensing image missing information decoupling reconstruction network DECRECNET, protecting the content information of the image by utilizing intrinsic content radiation consistency penalty, coordinating the imaging environment by imaging radiation penalty and imaging condition smoothness loss, and respectively performing targeted radiation adjustment on a foreground image and a background image by imaging radiation guidance and semantic guidance in a radiation guidance module. And finally, reconstructing the data set based on the constructed one-to-one paired missing information, and realizing missing information reconstruction.

Inventors

JIANG YONGHUA
LIU WEILING
LI ZHEN
ZHANG GUO
CUI HAO

Assignees

武汉大学

Dates

Publication Date: 20260505
Application Date: 20221024

Claims (8)

1. The method for reconstructing missing information of the remote sensing image under radiation decoupling is characterized by comprising the following steps of: S1, acquiring an image to be reconstructed and a reference image, and extracting a missing mask from the image to be reconstructed; S2, obtaining a synthetic image according to the image to be reconstructed, the reference image and the extracted missing mask, constructing a data set according to the synthetic image and the missing mask, and dividing a training data set; s3, constructing a remote sensing image missing information decoupling reconstruction network DECRECNET, DECRECNET which comprises an intrinsic content radiation correction module, a radiation guiding module, an imaging condition radiation correction module and a result output module, wherein the intrinsic content radiation correction module is used for obtaining intrinsic content components of a reconstructed output image according to input data and a semantic feature guiding diagram, the radiation guiding module comprises an imaging radiation guiding device and a semantic guiding device, the imaging radiation guiding device is used for obtaining an imaging radiation guiding feature diagram according to the input data, the semantic guiding device is used for obtaining a semantic feature guiding diagram according to the input data, the imaging condition radiation correction module is used for obtaining imaging condition components of the reconstructed output image according to the input data, the imaging condition radiation guiding feature diagram and the semantic feature guiding diagram, the result output module is used for calculating the intrinsic content components and the imaging condition components of the reconstructed output image based on the radiation decoupling model, the input data comprises a synthesized image and a missing mask, the imaging radiation guiding device is used for obtaining the imaging radiation guiding feature diagram according to the input data, the imaging radiation guiding feature diagram specifically comprises the steps of intercepting foreground and background areas in the feature diagram respectively through masking operation, calculating average sums of corresponding areas, and further transferring radiation conditions from the background to the foreground radiation guiding feature diagram by using a conversion technology, and obtaining imaging radiation variance differences; the semantic director obtains a semantic feature guiding image according to input data, wherein the semantic director particularly comprises the steps of continuing to encode an imaging radiation guiding feature image obtained by the imaging director, and constraining the same ground object in the front background through semantic constraint radiation consistency penalty to obtain the semantic guiding feature image; The method comprises the steps of S4, training a built remote sensing image missing information decoupling reconstruction network by using a training data set to obtain a trained remote sensing image missing information decoupling reconstruction network, wherein the method specifically comprises the steps of taking a synthesized image and a missing Mask as input, respectively obtaining an intrinsic content correction result and an imaging condition radiation correction result through an intrinsic content radiation correction module and an imaging condition radiation correction module, respectively intercepting foreground and background areas in a feature map by an imaging guide in a radiation guide module during the period, calculating the mean value and variance of the corresponding areas, transferring the radiation condition from the background to the foreground by using TRANSFERER technology, eliminating the radiation difference between the front background, continuing to encode the imaging guide feature map through a semantic guide in the radiation guide module, restraining the same kind of ground objects in the front background through semantic constraint radiation consistency penalty, and finally obtaining the reconstruction result according to the radiation decoupling model, wherein the total loss function adopted in the training process is as follows: (1) Wherein, the As a function of the overall loss, As an intrinsic content radiation consistency penalty function, In order to image the radiation penalty function, For the imaging condition radiation smoothing loss function, A consistency penalty function is radiated for the semantic constraint, A loss function is reconstructed for the image deletion information, 、、、、 Is a corresponding weighting factor for balancing the contributions of the different losses, and the calculation formula of each loss function is as follows: (2) in order to miss the intrinsic content component of the image, To output the intrinsic content component of the image after reconstruction, To output the gradients of the intrinsic content components of the image after reconstruction, Gradient for missing image; representing an encoder or decoder in the neural network calculation process; (3) 、 in order to output the conditional component of the image after reconstruction, For the image to be reconstructed, i.e. the missing image, Is a unit matrix; (4) 、 in order to output the gradient of the image condition component after reconstruction, (5) As a function of the similarity of the two, In order to obtain the characteristic diagram after downsampling, A signature of the radiation guide input directed for imaging, A feature map obtained after processing by an imaging condition transfer module in the imaging guided radiation guide; (6) In order to reconstruct the image that is output after the reconstruction, 、 Respectively representing gradients of the reconstructed output image and the image to be reconstructed; and S5, reconstructing the image to be reconstructed by using the trained remote sensing image missing information decoupling reconstruction network.
2. The method for reconstructing missing information of a remote sensing image under radiation decoupling as set forth in claim 1, wherein the step S1 of extracting the missing mask from the image to be reconstructed comprises: For the problem of information loss caused by cloud and cloud shadow, simulating the cloud and cloud shadow by using a Berlin noise method, and acquiring a cloud loss mask; for the problem of information missing caused by sensor faults, a strip missing mask is obtained by extracting a true missing strip; taking the obtained cloud missing mask and the obtained stripe missing mask as missing masks Wherein a region with a pixel value of 1 represents a missing region, and a region with a pixel value of 0 represents a non-missing region.
3. The method for reconstructing missing information of a remote sensing image under radiation decoupling according to claim 1, wherein in step S2, a composite image is obtained according to the image to be reconstructed, the reference image and the extracted missing mask, comprising: obtaining a background image according to the image to be reconstructed and the missing mask , wherein, In order for the image to be reconstructed, To be a missing mask; Acquiring a foreground image according to the reference image and the missing mask , wherein, Is a reference image; acquiring a composite image according to the acquired background image and foreground image , = 。
4. The method for reconstructing missing information of remote sensing images under radiation decoupling according to claim 1, wherein the intrinsic content radiation correction module of the network in step S3 takes input data and a semantic guidance feature map as inputs, performs semantic constraint on equivalent features in the front background, and obtains an intrinsic content component of the reconstructed output image.
5. The method for reconstructing missing information of a remote sensing image under radiation decoupling according to claim 1, wherein the imaging radiation correction module of the network in step S3 corrects the radiation difference in the front background by using the input data, the imaging radiation guide feature map and the semantic feature guide map as inputs to obtain imaging condition components of the reconstructed output image, including gain and offset.
6. A remote sensing image missing information reconstruction device under radiation decoupling, which is realized based on the method of claim 1, and comprises: the image acquisition module is used for acquiring an image to be reconstructed and a reference image, and extracting a missing mask from the image to be reconstructed; The data set construction module is used for obtaining a synthetic image according to the image to be reconstructed, the reference image and the extracted missing mask, constructing a data set according to the synthetic image and the missing mask and dividing a training data set; The network construction module is used for constructing a remote sensing image missing information decoupling reconstruction network DECRECNET, DECRECNET which comprises an intrinsic content radiation correction module, a radiation guiding module, an imaging condition radiation correction module and a result output module, wherein the intrinsic content radiation correction module is used for obtaining an intrinsic content component of a reconstructed output image according to input data and a semantic feature guiding image, the radiation guiding module comprises an imaging radiation guiding device and a semantic guiding device, the imaging radiation guiding device is used for obtaining an imaging radiation guiding feature image according to the input data, the semantic guiding device is used for obtaining a semantic feature guiding image according to the input data, the imaging condition radiation correction module is used for obtaining an imaging condition component of the reconstructed output image according to the input data, the imaging radiation guiding feature image and the semantic feature guiding image, the result output module is used for calculating the intrinsic content component and the imaging condition component of the reconstructed output image based on the radiation decoupling model to obtain the reconstructed image, and the input data comprises a synthesized image and a missing mask; The network training module is used for training the constructed remote sensing image missing information decoupling reconstruction network by utilizing the training data set to obtain a trained remote sensing image missing information decoupling reconstruction network; and the image reconstruction module is used for reconstructing the image to be reconstructed by using the trained remote sensing image missing information decoupling reconstruction network.
7. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when executed, implements the method according to any one of claims 1 to 5.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 5 when the program is executed.

Description

Remote sensing image missing information reconstruction method and device under radiation decoupling Technical Field The invention relates to the technical field of remote sensing, in particular to a remote sensing image missing information reconstruction method and device under radiation decoupling. Background The optical remote sensing image has the advantages of large width, high resolution and the like, and is an important means for earth observation. However, the optical satellite belongs to passive remote sensing, the imaging wave band is in the range of 0.38-0.76 um, and the satellite sensor often has the problem of information deletion caused by cloud and cloud shadow shielding, sensor faults and the like on images due to interference of atmospheric environment (haze, cloud and the like) and actual working conditions. For the problem of information loss caused by cloud shielding, 35% of the area of the global land range is covered by cloud layers every year, and the problem of image information loss caused by sensor faults is most typically Landsat-7 images, and 22% of scanning gaps appear in each Landsat-7 image since 2003, so that the use of the images is seriously affected. Therefore, the method has important significance and practical value for improving the data utilization rate of the remote sensing image by reconstructing the information of the information missing position in the optical remote sensing image. In recent years, due to strong nonlinear expression capability and strong generalization capability of deep learning, a deep learning method is gradually applied to the fields of optical remote sensing image super-resolution reconstruction, remote sensing image missing information reconstruction and the like, and then the existing method has the technical problem of poor reconstruction effect. Disclosure of Invention The invention provides a remote sensing image missing information reconstruction method under radiation decoupling, which is used for solving or at least partially solving the technical problem of poor reconstruction effect in the prior art. In order to solve the above technical problems, a first aspect of the present invention provides a method for reconstructing missing information of a remote sensing image under radiation decoupling, including: S1, acquiring an image to be reconstructed and a reference image, and extracting a missing mask from the image to be reconstructed; S2, obtaining a synthetic image according to the image to be reconstructed, the reference image and the extracted missing mask, constructing a data set according to the synthetic image and the missing mask, and dividing a training data set; S3, constructing a remote sensing image missing information decoupling reconstruction network DECRECNET, DECRECNET which comprises an intrinsic content radiation correction module, a radiation guiding module, an imaging condition radiation correction module and a result output module, wherein the intrinsic content radiation correction module is used for obtaining an intrinsic content component of a reconstructed output image according to input data and a semantic feature guiding diagram, the radiation guiding module comprises an imaging radiation guiding device and a semantic guiding device, the imaging radiation guiding device is used for obtaining an imaging radiation guiding feature diagram according to the input data, the semantic guiding device is used for obtaining a semantic feature guiding diagram according to the input data, the imaging condition radiation correction module is used for obtaining an imaging condition component of the reconstructed output image according to the input data, the imaging radiation guiding feature diagram and the semantic guiding feature diagram, the result output module is used for calculating the intrinsic content component and the imaging condition component of the reconstructed output image based on the radiation decoupling model to obtain the reconstructed image, and the input data comprises a synthesized image and a missing mask; s4, training the constructed remote sensing image missing information decoupling reconstruction network by using a training data set to obtain a trained remote sensing image missing information decoupling reconstruction network; and S5, reconstructing the image to be reconstructed by using the trained remote sensing image missing information decoupling reconstruction network. In one embodiment, in step S1, extracting the missing mask from the image to be reconstructed includes: for the problem of information loss caused by cloud and cloud shadow, simulating the cloud and cloud shadow by using a Berlin noise (Perlin noise) method to obtain a cloud loss mask; for the problem of information missing caused by sensor faults, a strip missing mask is obtained by extracting a true missing strip; And taking the obtained cloud missing mask and the obtained stripe missing mask as a missing mask M, wherein a r