CN-121982571-A - Multi-mode remote sensing data-based farmland surface disturbance and desertification monitoring method and system

CN121982571ACN 121982571 ACN121982571 ACN 121982571ACN-121982571-A

Abstract

The invention provides a farmland surface disturbance and desertification monitoring method and system based on multi-mode remote sensing, wherein the method comprises the steps of obtaining a remote sensing image, preprocessing, dividing the obtained multi-mode double-phase remote sensing image, constructing a Token sequence, inputting the Token sequence into a transducer backbone network for feature extraction, and obtaining optical mode features and SAR mode features; introducing a low-rank adapter into a network attention module, regulating gradient contribution based on a modal deviation score, constructing a time-phase-crossing differential change characteristic, fusing to obtain a fusion characteristic, generating a change probability map, applying consistency constraint of an optical and SAR prediction result, and outputting a final change detection result. The invention obviously improves the fusion equilibrium of the optical and SAR characteristics through a modal depolarization LoRA fine tuning mechanism, fully expresses the weak modal information, introduces cross-modal consistency constraint, and ensures consistency and credibility of a change detection result.

Inventors

WU ZIHANG
BAO TENGFEI
LU JIAYANG
CHANG YANHUI
FANG TAO
DONG FANLI
XIAO ZHIPENG
ZHANG SHUNPING

Assignees

上海交通大学内蒙古研究院

Dates

Publication Date: 20260505
Application Date: 20260128

Claims (10)

1. A farmland surface disturbance and desertification monitoring method based on multi-mode remote sensing is characterized by comprising the following steps: step S1, acquiring a remote sensing image and preprocessing the remote sensing image to obtain a multi-mode double-phase remote sensing image; The remote sensing image comprises an optical remote sensing image and an SAR remote sensing image; S2, dividing the multi-mode double-phase remote sensing image, constructing a Token sequence containing mode codes and phase codes, inputting the sequence into a transducer backbone network with shared weight for feature extraction, and obtaining optical mode features and SAR mode features; Step S3, introducing a mode independent low-rank adapter into an attention module of the transducer backbone network, and regulating gradient contribution based on a mode deviation score to perform mode depolarization on optical mode characteristics and SAR mode characteristics; s4, respectively constructing and fusing time-phase-crossing differential change characteristics for the depolarized optical mode characteristics and SAR mode characteristics to obtain fusion characteristics; and S5, generating a change probability map based on the fusion characteristics, applying consistency constraint of the optical and SAR prediction results, and outputting a final change detection result.
2. The method for monitoring disturbance and desertification of farmland surface based on multi-modal remote sensing as set forth in claim 1, wherein said transducer backbone network comprises an input embedded layer, a multi-headed self-attention layer, a feed-forward neural network layer and an output layer, and the model input is a multi-modal double-phase sample Change label map ; Wherein t 1 、t 2 represents a first time and a second time respectively; And Representing optical and SAR modes, respectively; representing an optical mode input image at time t 1 、t 2 , Representing an SAR modal input image at a time t 1 、t 2 ; And mapping the multi-mode double-phase remote sensing image to an embedded space through Patch division and Token construction, extracting features through a multi-layer network, and fusing and decoding to obtain a final change detection result.
3. The method for monitoring farmland surface disturbance and desertification based on multi-modal remote sensing according to claim 1, wherein on the basis of a VIT-L backbone network, a modal bias score MBS is defined according to a modal gradient norm difference ratio, and the index is used as a modal bias regularization term to weight total loss, a cross-modal de-bias deliberation module is constructed, the internal attention of a modality is attenuated through a bias correction matrix, and the cross-modal attention is enhanced, wherein MBS is defined as follows: Wherein, the And (3) with Respectively representing the related parameter sets of the optical mode and the SAR mode, and the corresponding gradient norms are And (3) with And to prevent the denominator from being zero, a small constant is introduced 。
4. The method for monitoring farmland surface disturbance and desertification based on multi-modal remote sensing as set forth in claim 1, wherein the optical prediction change map and the SAR prediction change map are respectively The consistency constraint loss is composed using a JS divergence term and a CD divergence term, formalized as follows: Wherein, the And The height and width of the input image are respectively, And Representing the optical and SAR modes respectively, The method comprises the steps of obtaining a prediction category, obtaining an average distribution of an optical prediction change map and an SAR prediction change map, obtaining a spatial position index u and v, wherein u represents a pixel index of an image in a horizontal direction, and v represents a pixel index of the image in a vertical direction.
5. The method for monitoring farmland surface disturbance and desertification based on multi-modal remote sensing as set forth in claim 1, wherein a total loss function is used in the training process of the model As an optimization target, a AdamW optimizer is adopted for training, and part of parameters of the trunk are frozen by utilizing a LoRA branch weight updating mode of the ViT-L trunk, so that only low-rank adapter parameters are trained; Total loss function The method comprises the following steps: Wherein the method comprises the steps of The task loss is detected for transformation, and the task loss is in a combination form of binary cross entropy loss and Dice loss; is a consistency constraint loss; Is the regular loss of the modal deviation, MBS is the modal deviation score; the JS and CD are Jensen-Shannon divergence and contrast divergence respectively; 、 is a super parameter, and is used for controlling a regular term.
6. A farmland surface disturbance and desertification monitoring system based on multi-mode remote sensing is characterized by comprising: the module M1 is used for acquiring a remote sensing image and preprocessing the remote sensing image to obtain a multi-mode double-phase remote sensing image; The remote sensing image comprises an optical remote sensing image and an SAR remote sensing image; dividing the multi-mode double-phase remote sensing image and constructing a Token sequence comprising mode codes and phase codes, inputting the sequence into a transducer backbone network with shared weight for feature extraction to obtain optical mode features and SAR mode features; a module M3, introducing a mode independent low-rank adapter into an attention module of the transducer backbone network, and regulating gradient contribution based on a mode deviation score to perform mode depolarization on optical mode characteristics and SAR mode characteristics; a module M4 respectively constructing and fusing time-phase-crossing differential change characteristics for the depolarized optical mode characteristics and SAR mode characteristics to obtain fusion characteristics; and a module M5, generating a change probability map based on the fusion characteristics, applying consistency constraint of the optical and SAR prediction results, and outputting a final change detection result.
7. The system for monitoring disturbance and desertification of farmland surface based on multi-modal remote sensing as set forth in claim 6, wherein said transducer backbone network comprises an input embedded layer, a multi-headed self-attention layer, a feed-forward neural network layer and an output layer, wherein the model input is a multi-modal bi-temporal sample Change label map ; Wherein t 1 、t 2 represents a first time and a second time respectively; And Representing optical and SAR modes, respectively; representing an optical mode input image at time t 1 、t 2 , Representing an SAR modal input image at a time t 1 、t 2 ; And mapping the multi-mode double-phase remote sensing image to an embedded space through Patch division and Token construction, extracting features through a multi-layer network, and fusing and decoding to obtain a final change detection result.
8. The system for monitoring disturbance and desertification of farmland surface based on multi-modal remote sensing according to claim 6, wherein on the basis of a VIT-L backbone network, a modal bias score MBS is defined according to a modal gradient norm difference ratio, and the index is used as a modal bias regularization term to weight total loss, a cross-modal de-bias deliberation module is constructed, the internal attention of a modality is attenuated through a bias correction matrix, and the cross-modal attention is enhanced, wherein MBS is defined as follows: Wherein, the And (3) with Respectively representing the related parameter sets of the optical mode and the SAR mode, and the corresponding gradient norms are And (3) with And to prevent the denominator from being zero, a small constant is introduced 。
9. The system for monitoring disturbance and desertification of farmland surface based on multi-modal remote sensing as set forth in claim 6, wherein the optical prediction change map and the SAR prediction change map are respectively The consistency constraint loss is composed using a JS divergence term and a CD divergence term, formalized as follows: Wherein, the And The height and width of the input image are respectively, And Representing the optical and SAR modes respectively, The method comprises the steps of obtaining a prediction category, obtaining an average distribution of an optical prediction change map and an SAR prediction change map, obtaining a spatial position index u and v, wherein u represents a pixel index of an image in a horizontal direction, and v represents a pixel index of the image in a vertical direction.
10. The system for monitoring disturbance and desertification of farmland based on multi-modal remote sensing as set forth in claim 6, wherein a total loss function is used during the training of the model As an optimization target, a AdamW optimizer is adopted for training, and part of parameters of the trunk are frozen by utilizing a LoRA branch weight updating mode of the ViT-L trunk, so that only low-rank adapter parameters are trained; Total loss function The method comprises the following steps: Wherein the method comprises the steps of The task loss is detected for transformation, and the task loss is in a combination form of binary cross entropy loss and Dice loss; is a consistency constraint loss; Is the regular loss of the modal deviation, MBS is the modal deviation score; Is a change prediction probability map, Y is a true change label map, JS and CD are Jensen-Shannon divergence and contrast divergence respectively, wherein 、 Is a super parameter, and is used for controlling a regular term.

Description

Multi-mode remote sensing data-based farmland surface disturbance and desertification monitoring method and system Technical Field The invention belongs to the technical field of remote sensing intelligent monitoring and ecological environment supervision, and particularly relates to a farmland surface disturbance and desertification monitoring method and system based on multi-mode remote sensing data, in particular to a multi-mode remote sensing monitoring method and system based on a modal deviation fine tuning mechanism and consistency constraint. Background Along with the increase of urban expansion, natural disaster frequency, ecological environment management and national resource management requirements and the continuous enhancement of remote sensing satellite observation capability, the remote sensing change detection technology becomes one of core means for monitoring the surface dynamic change, and is widely applied to the fields of disaster monitoring, ecological environment evaluation, urban expansion analysis and the like. In farmlands and surrounding areas, artificial activities such as illegal sand production can cause significant changes in surface morphology, texture structures and coverage types, thereby destroying cultivated land structures and exacerbating land degradation processes. The change is usually represented as a surface disturbance characteristic in the remote sensing image, and has similarity with desertification evolution in terms of spatial morphology and time sequence characteristics, so that unified modeling and change detection can be performed through a remote sensing monitoring means. The optical remote sensing image has the advantages of high spatial resolution and abundant texture and spectrum difference, and is the most widely applied data source in the current change detection. However, the optical sensor is significantly affected by the illumination condition and the atmospheric condition, and particularly has limited information under the cloud and fog shielding, rain and snow weather or night imaging conditions, so that stable and continuous dynamic monitoring is difficult to realize. In contrast, synthetic Aperture Radar (SAR) images have all-day and all-weather imaging capability, can penetrate through cloud and smoke dust, and provide reliable supplement for change extraction in complex environments. In recent years, with the rapid development of deep learning methods, convolutional neural networks, two-branch structure models, and remote sensing change detection models based on transformers have become the main technical routes for change detection. Part of the research starts to explore the combined feature learning of the optical image and the SAR image so as to exert the complementary advantages of the optical image and the SAR image. However, the existing multi-mode change detection method still has the following problems: (1) The texture information of the optical image is rich, and the optical image is often dominant in model training, so that SAR characteristic contribution is insufficient, and serious modal deviation problem occurs; (2) The existing fusion process is mostly data driven, lacks unified modeling of physical differences between optical reflection and radar backscattering, and lacks consistency constraint in output; (3) Aiming at specific downstream tasks, the full-parameter training Vision Transformer (ViT) multi-mode network has large calculation overhead, and is not beneficial to edge deployment or rapid migration learning. Therefore, the method can effectively inhibit modal deviation, enhance consistency of optical and SAR decisions, has low-cost fine tuning capability, and improves reliability and practicality of surface disturbance monitoring in complex scenes so as to realize the high-precision and strong-robustness change detection technology in complex environments. Disclosure of Invention Aiming at the defects in the prior art, the invention aims to provide a farmland surface disturbance and desertification monitoring method and system based on multi-mode remote sensing data. The invention provides a farmland surface disturbance and desertification monitoring method based on multi-mode remote sensing data, which comprises the following steps: step S1, acquiring a remote sensing image and preprocessing the remote sensing image to obtain a multi-mode double-phase remote sensing image; The remote sensing image comprises an optical remote sensing image and an SAR remote sensing image; S2, dividing the multi-mode double-phase remote sensing image, constructing a Token sequence containing mode codes and phase codes, inputting the sequence into a transducer backbone network with shared weight for feature extraction, and obtaining optical mode features and SAR mode features; Step S3, introducing a mode independent low-rank adapter into an attention module of the transducer backbone network, and regulating gradient contribution based on