CN-121810508-B - SAR-to-optical image conversion method and system based on space-frequency guidance

CN121810508BCN 121810508 BCN121810508 BCN 121810508BCN-121810508-B

Abstract

The invention discloses a space-frequency guide-based SAR-to-optical image conversion method and a system, which belong to the technical field of image conversion, wherein the method comprises the steps of extracting image characteristics of SAR images; extracting noise from an optical image to obtain a target noise image, carrying out feature extraction on the target noise image to obtain a noise feature, dividing the noise feature into a processing path feature and a shortcut path feature along a channel dimension, carrying out residual connection on the image feature and the processing path feature to obtain a residual fusion feature, carrying out enhancement on the residual fusion feature to obtain an enhanced feature, carrying out splicing fusion on the enhanced feature and the shortcut path feature, adding the enhanced feature and the noise feature to obtain an injection feature, decoding the injection feature to obtain a predicted noise image, and subtracting the predicted noise image from the target noise image to obtain a virtual optical image. The method improves the generation quality and the calculation rate of the virtual optical image and solves the problem of poor quality of the virtual image generated by the current method.

Inventors

LIU BO
XING YINAN
ZHANG YUNFENG
Ren Yuehe

Assignees

山东财经大学

Dates

Publication Date: 20260505
Application Date: 20260310

Claims (10)

1. A space-frequency guided SAR to optical image conversion method comprising: Acquiring SAR images and optical images; extracting image features of the SAR image; extracting noise in the optical image to obtain a target noise image; Extracting features of the target noise image to obtain noise features; dividing the noise characteristics into processing path characteristics and shortcut path characteristics along the channel dimension; Residual connection is carried out on the image features and the processing path features, so that residual fusion features are obtained; enhancing the residual fusion characteristic to obtain an enhanced characteristic; the enhanced features and the shortcut path features are spliced and fused and added with noise features to obtain injection features; decoding the injection features to obtain a prediction noise image; and subtracting the predicted noise image from the target noise image to obtain a virtual optical image.
2. The space-frequency guided SAR to optical image conversion method according to claim 1, wherein the image features are up-sampled, followed by weighted averaging after light weight linear layer projection to obtain fusion condition features; And adding the fusion condition features and the processing path features to obtain residual fusion features.
3. The spatial-frequency guided SAR-to-optical image conversion method according to claim 1, wherein the residual fusion feature is channel-attention and spatial-attention modulated to obtain an enhanced feature.
4. The space-frequency guided SAR to optical image conversion method according to claim 1, wherein the multi-scale spatial features and multi-scale frequency domain features of the SAR image are extracted; and fusing the multi-scale space features and the multi-scale frequency domain features to obtain the image features of the SAR image.
5. The space-frequency guided SAR to optical image conversion method according to claim 4, wherein the SAR image is converted into a single channel gray scale image; Acquiring a complex frequency spectrum of a single-channel gray level image at a frequency coordinate; calculating an amplitude spectrum of the complex frequency spectrum, and carrying out logarithmic transformation on the amplitude spectrum to obtain a logarithmic amplitude spectrum; Obtaining a frequency image of three channels according to the logarithmic magnitude spectrum; And carrying out multi-scale feature extraction on the frequency images of the three channels to obtain multi-scale frequency domain features of the SAR image.
6. The space-frequency guided SAR to optical image conversion method according to claim 1, wherein the target noise image is encoded to obtain an encoding characteristic; and further extracting the coding features to obtain noise features.
7. A space-frequency guided SAR to optical image conversion system comprising: an image acquisition unit configured to acquire an SAR image and an optical image; the image feature extraction unit is used for extracting image features of the SAR image; The noise characteristic extraction unit is used for extracting noise in the optical image to obtain a target noise image; The image processing device comprises a diffusion unit, a target noise image and a virtual optical image, wherein the diffusion unit is used for dividing noise characteristics into processing path characteristics and shortcut path characteristics along a channel dimension, carrying out residual connection on the image characteristics and the processing path characteristics to obtain residual fusion characteristics, carrying out enhancement on the residual fusion characteristics to obtain enhanced characteristics, carrying out splicing fusion on the enhanced characteristics and the shortcut path characteristics, adding the enhanced characteristics and the noise characteristics to obtain injection characteristics, decoding the injection characteristics to obtain a predicted noise image, and subtracting the predicted noise image from the target noise image to obtain the virtual optical image.
8. An electronic device, the device comprising: A processor adapted to execute a computer program; A computer readable storage medium having stored therein a computer program which, when executed by the processor, implements the space-frequency guided SAR-to-optical image conversion method of any one of claims 1-6.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor and to perform the spatial-frequency guided SAR-to-optical image conversion method according to any of claims 1-6.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the space-frequency guided SAR-to-optical image conversion method according to any of claims 1-6.

Description

SAR-to-optical image conversion method and system based on space-frequency guidance Technical Field The invention relates to the technical field of remote sensing image processing and computer vision, in particular to a SAR-to-optical image conversion method and system based on space-frequency guidance. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. With the rapid development of remote sensing technology, synthetic Aperture Radar (SAR) and optical remote sensing images are increasingly used in the fields of environmental monitoring, disaster assessment, military reconnaissance and the like. Optical images are favored by abundant texture information and good visual interpretability, but the acquisition of the optical images is influenced by factors such as weather, illumination and the like, so that all-weather continuous observation is difficult to realize. In contrast, SAR has imaging capability all over the day and around the clock, and can effectively make up for the deficiency of optical data. Therefore, SAR-to-optical (S2O) image conversion techniques are a research hotspot in the remote sensing field, aiming at converting SAR images into high-quality optical images, so as to support more accurate ground feature recognition and environmental understanding. However, SAR images are severely affected by multiplicative speckle noise, the signal-to-noise ratio is low, the noise is spatially coupled with the real signal, resulting in difficulties in feature extraction and cross-modal alignment in conventional methods, affecting the structural integrity and detail fidelity of the transformed image. In recent years, diffusion models have shown significant potential in image generation and translation tasks due to their strong generation capacity and training stability. However, in the existing SAR image-to-optical image conversion method based on the diffusion model, the SAR image and the noise image are simply spliced and injected, so that effective cross-modal alignment is difficult to realize, structural distortion and texture confusion are easy to cause, and finally the quality of the generated virtual optical image is poor. Disclosure of Invention In order to solve the problems, the invention provides a SAR-to-optical image conversion method and a SAR-to-optical image conversion system based on space-frequency guidance, which improve the quality of a finally obtained virtual optical image. In order to achieve the above purpose, the invention adopts the following technical scheme: in a first aspect, the present invention proposes a spatial-frequency guided SAR to optical image conversion method comprising: Acquiring SAR images and optical images; extracting image features of the SAR image; extracting noise in the optical image to obtain a target noise image; Extracting features of the target noise image to obtain noise features; dividing the noise characteristics into processing path characteristics and shortcut path characteristics along the channel dimension; Residual connection is carried out on the image features and the processing path features, so that residual fusion features are obtained; enhancing the residual fusion characteristic to obtain an enhanced characteristic; the enhanced features and the shortcut path features are spliced and fused and added with noise features to obtain injection features; decoding the injection features to obtain a prediction noise image; and subtracting the predicted noise image from the target noise image to obtain a virtual optical image. Further, up-sampling the image features, and then obtaining fusion condition features through weighted average after light linear layer projection; And adding the fusion condition features and the processing path features to obtain residual fusion features. Further, channel attention and spatial attention modulation is performed on the residual fusion feature, and an enhanced feature is obtained. Further, extracting multi-scale space features and multi-scale frequency domain features of the SAR image; and fusing the multi-scale space features and the multi-scale frequency domain features to obtain the image features of the SAR image. Further, converting the SAR image into a single-channel gray scale image; Acquiring a complex frequency spectrum of a single-channel gray level image at a frequency coordinate; calculating an amplitude spectrum of the complex frequency spectrum, and carrying out logarithmic transformation on the amplitude spectrum to obtain a logarithmic amplitude spectrum; Obtaining a frequency image of three channels according to the logarithmic magnitude spectrum; And carrying out multi-scale feature extraction on the frequency images of the three channels to obtain multi-scale frequency domain features of the SAR image. Further, encoding the target noise image to obtain encoding characteristics; and fur