CN-122023174-A - Remote sensing image multitasking restoration method based on layered dynamic prompt

CN122023174ACN 122023174 ACN122023174 ACN 122023174ACN-122023174-A

Abstract

The invention belongs to the technical field of remote sensing image processing and computer vision, and provides a remote sensing image multitasking method based on layered dynamic prompt, which aims to perform dynamic characteristic modulation according to an input image state, massive paired clear-degradation training samples and accurate task labels are generated in real time based on a sensor imaging physical model by introducing an online degradation simulation mechanism, and a layered prompt modulation network is combined to dynamically adjust a network characteristic extraction mode by utilizing the task labels. Therefore, under the condition of no need of manual labeling, the automatic sensing and high-quality restoration of various degradation phenomena such as low-frequency chromatic aberration, vertical stripes, mixed noise and the like in the high-resolution remote sensing image are realized.

Inventors

ZHOU KAI
FAN LIMING
GONG LIN
TANG YAO
LI XINFANG
JIA HONGGUANG

Assignees

长光卫星技术股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260128

Claims (8)

1. The remote sensing image multi-task restoration method based on layered dynamic prompt is characterized by comprising the following steps of: Step 1, synthesizing corresponding degradation images and task labels in real time by taking original clear images as references, and generating degradation samples according to degradation simulation strategies corresponding to the task labels so as to construct an online degradation training sample library; Step 2, training a recovery network model based on layered prompt modulation according to the online degradation training sample library to realize self-adaptive recovery; Step 3, judging the degradation type of the image to be restored, if the degradation type is the high-frequency local degradation characteristic, executing step 4, and if the degradation type is the low-frequency offset color problem, executing step 5; step4, aiming at the high-frequency local degradation characteristics, adopting a weighted overlapping sliding window strategy to perform full resolution reasoning, and eliminating a blocking boundary effect while recovering details to obtain a restored image; and step 5, aiming at the low frequency offset problem, adopting a one-dimensional cumulative gain alignment algorithm based on downsampling to obtain a restored image.
2. The method of claim 1, wherein the degradation simulation strategy in step 1 includes low frequency radiation distortion and color difference simulation, high frequency streak and non-uniform response simulation, and sensor noise and mixed random noise simulation.
3. The remote sensing image multi-task restoration method based on layered dynamic prompt according to claim 2 is characterized in that the specific processing flow of low-frequency radiation distortion and color difference simulation is as follows, firstly, smooth optical gradual change signals are generated through Gaussian filtering, step signals with hard edges are generated through combination of random break points, the optical gradual change signals and the step signals are overlapped through random weighting, preset dynamic intensity scaling is applied, a non-uniform gain curve which accords with physical reality is constructed, the non-uniform gain curve acts on an original image in a band-by-band mode, and a low-frequency color difference degradation sample is generated.
4. The remote sensing image multi-task restoration method based on layered dynamic prompt according to claim 2, wherein the specific processing flow of the high-frequency stripe and the non-uniform response simulation is as follows, gain coefficient is generated by utilizing truncated normal distribution random sampling Bias coefficient And constructing a multi-scale coupling mechanism, and simulating a high-frequency fringe degradation sample by acting the generated fringe parameter matrix on the original image.
5. The remote sensing image multi-task restoration method based on layered dynamic prompt according to claim 2 is characterized in that the specific processing flow of sensor inherent noise and mixed random noise simulation is as follows, a dynamic noise intensity range is set according to the signal-to-noise ratio characteristic of a sensor, noise standard deviation parameters are randomly sampled from the range, a random noise tensor conforming to zero-mean Gaussian distribution is generated, and the random noise tensor is superimposed on an original image in an additive mode to generate a random noise degradation sample.
6. The method of claim 1, wherein the restoring network model comprises a hierarchically symmetric backbone network for capturing long-range pixel dependencies, and for generating dynamic modulation parameter scaling factors Bias factor The attention mechanism module comprises a gating modulation and a self-attention modulation: The self-attention modulation uses modulation parameters to carry out affine transformation on the query Q, the key K and the value V projection, the self-adaptation adjustment of the weights of different channels when calculating attention force diagram, Wherein the method comprises the steps of Representing element-wise multiplication, the final attention output is calculated as: ; The gating modulation utilizes modulation parameters to specially weight gating branches, and a soft start Tanh gating mechanism is designed: , , Wherein, the To initialize to a scientific scaling factor of 1, By means of And Modulation gating branching Weighting: and finally, the output is obtained by the product of the activated characteristic branch and the modulated gate control: Wherein, the Is GELU activated.
7. The remote sensing image multi-task restoration method based on layered dynamic prompt is characterized in that step 4 is firstly conducted with self-adaptive filling and grid division, an inference grid is generated according to set size and overlapping rate, secondly, each local block after sliding window is conducted with block inference and weighted fusion, the system reads image data block by block, inputs the image data into a pre-training network to conduct forward propagation, outputs a high-frequency restoration result, a weight mask sensitive to spatial positions is introduced, weights are linearly attenuated from the center to the edge in an overlapping area, weighted prediction values and weight sums of all blocks are recorded by utilizing an accumulation buffer area, and finally normalization division is conducted to achieve seamless splicing, so that edge transition smoothness and block mosaic are completed.
8. The remote sensing image multi-task restoration method based on layered dynamic prompt according to claim 1 is characterized in that the step 5 is firstly conducted with downsampling, sliding window reasoning is conducted on a feature map after downsampling, a low-frequency gain field is predicted, a two-dimensional gain map is conducted with statistical aggregation and conversion to a one-dimensional gain curve along a column direction, then robust accumulation alignment is conducted, median filtering is adopted, relative offset among blocks is estimated, gain curves of all blocks are aligned to the same datum plane through accumulation addition, a global continuous gain field is formed, upsampling and correction are conducted finally, the aligned global gain field is conducted with bilinear interpolation to be sampled back to an original image resolution, and the gain field is overlapped to the original image, so that low-frequency removal processing is completed.

Description

Remote sensing image multitasking restoration method based on layered dynamic prompt Technical Field The invention belongs to the technical field of remote sensing image processing and computer vision. Background The remote sensing image restoration aims to restore clear and high-fidelity images from data with residual problems (color difference, stripes, noise and the like) after quality degradation or pretreatment, and is a key link for improving the usability of remote sensing data. The current few 'all-in-one' restoration methods mostly adopt a static weight sharing mechanism, namely, no matter what degradation type the input image is, the network adopts a fixed convolution kernel and attention weight for processing. However, the focus of different restoration tasks on frequency and features is quite different, e.g. de-color difference requires low frequency global information, whereas de-striping and de-noising requires high frequency local texture. The existing method lacks a mechanism capable of carrying out dynamic characteristic modulation according to the state of an input image, so that a model cannot 'sense' the current task demand and cannot flexibly switch the characteristic extraction mode among different tasks, and the improvement of recovery precision is limited. Disclosure of Invention Aiming at the technical problems, the invention provides a remote sensing image multi-task restoration method based on layered dynamic prompt, which comprises the following specific technical scheme: Step 1, synthesizing corresponding degradation images and task labels in real time by taking original clear images as references, and generating degradation samples according to degradation simulation strategies corresponding to the task labels so as to construct an online degradation training sample library; Step 2, training a recovery network model based on layered prompt modulation according to the online degradation training sample library to realize self-adaptive recovery; Step 3, judging the degradation type of the image to be restored, if the degradation type is the high-frequency local degradation characteristic, executing step 4, and if the degradation type is the low-frequency offset color problem, executing step 5; step4, aiming at the high-frequency local degradation characteristics, adopting a weighted overlapping sliding window strategy to perform full resolution reasoning, and eliminating a blocking boundary effect while recovering details to obtain a restored image; and step 5, aiming at the low frequency offset problem, adopting a one-dimensional cumulative gain alignment algorithm based on downsampling to obtain a restored image. The technical effects are as follows: compared with the existing special restoration algorithm or the traditional artificial image enhancement means, the method has the following remarkable beneficial effects: The sample construction cost is remarkably reduced, and the model generalization capability is improved. Unlike traditional methods that rely on manual collection and labeling of paired samples, the present invention employs an on-line degradation simulation strategy based on a physical mechanism. In the training stage, massive clear-degradation training data covering low-frequency chromatic aberration, vertical stripes and mixed noise can be synthesized in real time based on the original clear image without any manual labeling cost, so that the robustness and generalization capability of the model in coping with unknown complex degradation are improved. The invention constructs a layered prompt modulation network and utilizes a task perception global prompt mechanism to ensure that a single weight-sharing deep learning model has the capabilities of color difference removal, stripe removal and noise removal, reduces the storage space requirement and the video memory occupation of algorithm deployment, and is suitable for resource-limited scenes such as satellite on-board processing or edge computing equipment. The invention introduces a characteristic linear modulation and soft start gating mechanism and endows a static network with dynamic 'perception' capability. The network can adaptively adjust the channel weight and frequency response characteristics of the self-attention mechanism according to the input task instructions and the degradation statistical characteristics extracted in real time in the reasoning stage. In summary, the invention introduces a prompt modulation mechanism, and generates dynamic modulation parameters by extracting inherent degradation characteristics of images and combining with explicit task instructions. The self-attention mechanism and the feedforward network in the deep layer of the network are subjected to characteristic level linear modulation by utilizing the parameters, so that a single model can adaptively adjust characteristic response and a concerned region according to specific task labels and degradation degrees, and the