CN-122023902-A - Remote sensing change detection method and system based on deep reinforcement learning

CN122023902ACN 122023902 ACN122023902 ACN 122023902ACN-122023902-A

Abstract

The invention provides a remote sensing change detection method and a remote sensing change detection system based on deep reinforcement learning, wherein the method comprises the steps of carrying out feature extraction and difference modeling on an input multi-temporal remote sensing image through a deep learning network so as to generate a preliminary change prediction graph; and introducing a reinforcement learning module, taking the preliminary change prediction graph as an initial state, and performing multi-round iterative optimization on the preliminary change prediction graph through a decision process comprising a state space, an action space and a reward function so as to generate a final change detection result. And the reward function comprehensively considers the cross-over ratio lifting value, the F1 score lifting value and the cross entropy loss, and introduces a false positive punishment strengthening mechanism to repair the missing report and the false alarm region preferentially. According to the invention, through the iterative optimization of reinforcement learning, errors in the primary detection result can be effectively corrected, and the detection precision and robustness are remarkably improved.

Inventors

BAO TENGFEI
ZHAO YAN
LU JIAYANG
CHANG YANHUI
XIAO ZHIPENG
ZHANG SHUNPING
DONG FANLI
FANG TAO

Assignees

上海交通大学内蒙古研究院

Dates

Publication Date: 20260512
Application Date: 20260129

Claims (10)

1. The remote sensing change detection method based on deep reinforcement learning is characterized by comprising the following steps of: Performing multi-scale depth feature extraction and difference comparison on the paired multi-temporal remote sensing images through a deep learning network to generate a preliminary change prediction graph and a multi-scale feature pyramid, wherein the preliminary change prediction graph represents the probability of change of each pixel position, and the multi-scale feature pyramid comprises feature graphs containing context information extracted by different network depths; And introducing a reinforcement learning module, taking the preliminary change prediction graph and the multi-scale feature pyramid as initial states, and performing multi-round iterative optimization on the preliminary change prediction graph through a decision process comprising a state space, an action space and a reward function to generate a final change detection result.
2. The method for detecting remote sensing changes based on deep reinforcement learning according to claim 1, wherein the step of generating a preliminary change prediction graph through a deep learning network comprises: adopting a twin network structure, wherein the twin network comprises encoders sharing weights, and the multi-scale characteristics of the multi-temporal remote sensing images are respectively extracted through the encoders; performing feature difference modeling on the extracted multi-scale features to obtain difference features reflecting image changes; and inputting the difference characteristics to a decoder, performing up-sampling and characteristic fusion processing by the decoder, and outputting the preliminary change prediction graph.
3. The method for detecting remote sensing change based on deep reinforcement learning according to claim 1, wherein the input state of the reinforcement learning module is composed of three parts: extracting by an encoder of the deep learning network, and processing a deep feature pyramid by a multi-scale feature fusion module; the segmentation prediction result of the previous iteration is adopted, wherein the preliminary change prediction graph is adopted as the segmentation prediction result in the first iteration; The difference graph between the predicted result and the real label of the previous round is used for guiding the strategy network to focus on the change boundary and the difficult and difficult area.
4. The method for detecting remote sensing change based on deep reinforcement learning according to claim 1, wherein the reinforcement learning module guides the iterative optimization process through a reward function R; the definition of the reward function R is: Wherein, the And Respectively represent the cross-ratio and F1 score after the optimization action is executed, and And Then CrossEntropyLoss is the cross entropy loss term for the cross ratio and F1 score before executing the optimization action, and α, β, γ are preset weight super parameters.
5. The remote sensing change detection method based on deep reinforcement learning according to claim 4, wherein a punishment strengthening mechanism is introduced into the reward function, different weights are given to the misclassified pixels through a position weight map, and punishment weights of pixels in a missing report area and a false alarm area are greater than those of pixels in a correct classification; the missing report area is an area which is truly changed but predicted to be unchanged; the false alarm region is a region that is truly unchanged but predicted to be changed.
6. The method for detecting remote sensing change based on deep reinforcement learning according to claim 1, wherein a staged training strategy is adopted, specifically comprising: the first step, independently training the deep learning network until the model converges; Secondly, freezing encoder parameters of the deep learning network, and starting training of the strategy network in the reinforcement learning module; And thirdly, carrying out joint fine adjustment on the whole model, and improving the whole performance of the model.
7. A remote sensing change detection system based on deep reinforcement learning, comprising: The data preprocessing module is used for carrying out normalization and dimension unification operation on the input multi-temporal remote sensing image; the deep learning prediction module is internally provided with a twin network structure and is used for extracting image features and generating a preliminary change prediction graph; the reinforcement learning optimization module is used for carrying out iterative optimization on the preliminary change prediction graph and outputting a final change detection result; the storage module is used for storing the trained model parameters and the remote sensing data set; and the processing unit is used for executing the calculation tasks of the modules.
8. The deep reinforcement learning based remote sensing change detection system of claim 7, wherein the reinforcement learning optimization module comprises: the state construction unit is used for constructing an input state containing depth characteristics, a previous round of prediction results and a difference graph; The multi-scale feature fusion unit is used for fusing and refining the multi-scale features output by the deep learning prediction module; A policy network for outputting a classification adjustment action of the pixel level according to the input state; and the rewarding calculation unit is used for calculating rewarding values according to a preset rewarding function and guiding the training of the strategy network.
9. The remote sensing change detection system based on deep reinforcement learning according to claim 7, wherein the normalization operation of the data preprocessing module adopts a mean value-standard deviation normalization method, and the dimension unification operation is realized by an interpolation method, so that the resolution of the multi-temporal remote sensing images is consistent.
10. The system for detecting remote sensing changes based on deep reinforcement learning according to claim 7, wherein the decoder of the deep learning prediction module adopts a transposed convolution layer to realize upsampling, and fuses multi-scale features output by the encoder in a jump connection mode, so as to improve the detail representation capability of the preliminary change prediction graph.

Description

Remote sensing change detection method and system based on deep reinforcement learning Technical Field The invention relates to the technical field of remote sensing image processing, in particular to a remote sensing change detection method and system based on deep reinforcement learning. Background The remote sensing change detection technology is widely applied to the fields of urban expansion monitoring, ecological environment assessment, disaster response, resource management and the like as an important means for identifying earth surface coverage or land utilization change by utilizing multi-temporal remote sensing images. Along with the improvement of the resolution ratio of the remote sensing image and the shortening of the observation period, the feature change presents more detailed and time sequence complex characteristics, and particularly when facing complex background, tiny change and irregular boundary, the traditional change detection method gradually exposes the problems of low precision, poor generalization capability and the like. The traditional method generally depends on fixed characteristic extraction and judgment rules, is difficult to adapt to various heterogeneous data and complex environments, and particularly faces great challenges in the practical application of high-precision remote sensing change detection. At present, a change detection method based on deep learning, in particular to Convolutional Neural Networks (CNN), U-Net, siamese networks and the like, has made remarkable progress in pixel-level semantic segmentation tasks. The methods can automatically extract features from the images and perform end-to-end region extraction. For example, in the prior art, a manner of constructing a twin network structure is often adopted to effectively extract depth features of multi-phase images and perform difference comparison, so as to realize preliminary positioning of a change region. However, these deep learning-based approaches still face a series of challenges in practical applications. Firstly, the rolling and pooling operation in the network inevitably loses the space detail information of the image, so that the boundary segmentation of the model on the change area is not fine enough, and the problems of edge blurring, saw tooth and the like often exist in the final detection result, thereby influencing the detection precision. Secondly, the remote sensing image is extremely easy to be interfered by non-ground object real change factors such as illumination change, season replacement, cloud cover and the like, and the factors can form pseudo-change on the image, and the existing method has low robustness to the noise, is easy to generate false alarm, and is capable of misjudging an unchanged area as a changed area. Again, in the face of small target changes, morphological changes of non-rigid targets (e.g., rivers, lakes), and areas of complex background, the recognition ability of existing models can be significantly reduced, and false negatives, i.e., true change areas are missed, can easily occur. Finally, most of the existing methods adopt an 'end-to-end' mode of one-time forward propagation to directly output a detection result, lack iterative optimization processes similar to repeated review and correction by human experts, and are difficult to finely adjust and correct difficult and wrong areas in the primary detection result. In general, these methods rely on static training, lack of interaction with the external environment, and cannot dynamically adjust the prediction strategy, and especially when there is fuzzy boundary, pseudo-variation or heterogeneous data, the model is prone to false detection and omission, which limits the stability and universality of the model in practical remote sensing applications. In order to improve the capability of detecting changes in a complex scene, in recent years, some researches have introduced reinforcement learning methods. Reinforcement learning can continuously optimize decision strategies on the basis of feedback rewards by constructing an interaction mechanism of the intelligent body and the environment, and breaks through the limitation of static prediction-single supervision of a traditional deep learning model. Deep reinforcement learning (Deep Reinforcement Learning, DRL) combines the advantages of reinforcement learning and deep neural networks, and particularly shows strong dynamic learning capability and strategy migration capability in tasks such as image segmentation, path planning, intelligent decision-making and the like. Although deep reinforcement learning has made a remarkable progress in the fields of image processing and remote sensing, research for effectively applying reinforcement learning to remote sensing change detection is still under exploration. The existing research has some problems to be solved, including unreasonable task modeling, insufficient state space design and lack of pertinence of a r