CN-121998854-A - Night vision image noise reduction method based on double-domain joint model

CN121998854ACN 121998854 ACN121998854 ACN 121998854ACN-121998854-A

Abstract

The invention relates to the technical field of digital image processing, and discloses a night vision image noise reduction method based on a double-domain joint model, which comprises the following steps of obtaining an original Bayer domain image; the method comprises the steps of inputting the two noise reduction images into a time domain noise reduction model and a frequency domain noise reduction model in parallel to obtain a time domain noise reduction image and a frequency domain noise reduction image respectively, wherein the time domain model is a convolutional neural network, random noise is restrained in a space domain, the frequency domain model filters periodic noise in a frequency domain through learning a frequency domain mask, weighting is dynamically calculated according to a feature map before fusion of the two models through an attention fusion mechanism, the two noise reduction images are subjected to weighted fusion to generate a target noise reduction image, and the models perform end-to-end training by adopting a global loss function comprising time domain loss, frequency domain loss and domain alignment consistency loss. The invention effectively solves the contradiction between mixed noise suppression and detail retention, and improves the definition and visual quality of night vision images.

Inventors

SHI JIAN
HUANG GUODONG
ZHOU CHUNYANG
WANG LI
LIU XIAOXIN
CHEN WEI
QIAN WEI
LIN SHIKUN
YAN BAIHUI
HUANG HUABIN
MA JINTAO
JIANG HAO
SUN XIANHE

Assignees

南方电网绿能科技(广东)有限公司

Dates

Publication Date: 20260508
Application Date: 20251215

Claims (10)

1. A night vision image noise reduction method based on a dual-domain joint model is characterized by comprising the following steps: S1, acquiring an original Bayer domain image to be processed; S2, inputting the original Bayer domain image into a preset time domain noise reduction model to obtain a time domain noise reduction image; s3, inputting the original Bayer domain image into a preset frequency domain noise reduction model to obtain a frequency domain noise reduction image; s4, carrying out fusion processing on the time domain noise reduction image and the frequency domain noise reduction image to generate a target noise reduction image.
2. The night vision image denoising method based on the two-domain joint model according to claim 1, wherein step S2 specifically comprises: The time domain noise reduction model is configured to process the original Bayer domain image in a spatial dimension based on a convolutional neural network to obtain the time domain noise reduction image.
3. The night vision image denoising method based on the two-domain joint model according to claim 1, wherein step S3 specifically comprises: The frequency domain noise reduction model is configured to perform fourier transform on the original Bayer domain image to obtain a frequency domain representation of the original Bayer domain image, perform a filtering operation of element-by-element multiplication on the frequency domain representation by learning a frequency domain filtering mask, and perform inverse fourier transform on the frequency domain representation after the filtering operation to obtain the frequency domain noise reduction image.
4. The night vision image denoising method based on the two-domain joint model according to claim 1, wherein step S4 specifically comprises: dynamically calculating attention weight according to the feature map of the time domain noise reduction model and the frequency domain noise reduction model before fusion; Performing weighted fusion on the time domain noise reduction image and the frequency domain noise reduction image by using the attention weight to generate the target noise reduction image; The two feature maps are respectively output by the last layer of network before the fusion of the time domain noise reduction model and the frequency domain noise reduction model.
5. The night vision image denoising method based on a two-domain joint model according to claim 4, wherein dynamically calculating the attention weight comprises: Splicing the feature images of the time domain noise reduction model and the frequency domain noise reduction model before fusion to obtain a spliced feature image; And after the spliced feature map is input into a weight matrix for calculation, generating the attention weight through a Sigmoid activation function.
6. The night vision image denoising method based on the dual-domain joint model according to claim 1, wherein the time domain denoising model and the frequency domain denoising model are obtained through joint training in an end-to-end mode, and the joint training process is optimized by adopting a global loss function.
7. The night vision image denoising method based on a two-domain joint model according to claim 6, wherein the global loss function comprises: a weighted sum of the time domain loss, the frequency domain loss, and the domain alignment consistency loss; Wherein the temporal loss comprises an L1 loss between the temporal noise reduction image and a corresponding clear label image, and a structural similarity SSIM loss between the temporal noise reduction image and the clear label image; the frequency domain loss comprises the step of calculating the L1 loss between the frequency domain noise reduction image and the clear label image on the frequency domain after Fourier transformation is carried out on the frequency domain noise reduction image and the clear label image respectively.
8. The night vision image denoising method based on the dual-domain joint model according to claim 7, wherein the domain alignment consistency loss is calculated by the following steps: Carrying out global average pooling operation on the feature images of the time domain noise reduction model and the frequency domain noise reduction model before fusion respectively; And calculating the L2 loss between two results obtained by the global average pooling operation, and obtaining the domain alignment consistency loss.
9. The night vision image denoising method based on a two-domain joint model according to claim 6, wherein the joint training adopts a mixed data set, and the mixed data set comprises an image selected from SIDD, DND, RENOIR data sets and a self-acquired night vision image.
10. The night vision image denoising method based on a two-domain joint model according to claim 1, further comprising a step of calibrating an imaging system before the original Bayer domain image is acquired in step S1, wherein the calibrating comprises: Black level calibration, noise characteristic calibration, static dead point calibration, lens dark angle calibration, white balance coefficient calibration and color correction matrix calibration.

Description

Night vision image noise reduction method based on double-domain joint model Technical Field The invention relates to the technical field of digital image processing, in particular to a night vision image noise reduction method based on a dual-domain joint model. Background Night vision images are collected under extremely low illumination conditions, the signal to noise ratio is low, and complex mixed noise formed by sensor thermal noise, circuit readout noise, external electromagnetic interference and the like is commonly present, so that the visual quality and the legibility of the images are seriously reduced. Existing night vision image denoising methods typically process in a single domain. Although the method based on the space domain or the time domain can process the noise which is randomly distributed, the periodic noise or the fixed mode noise is difficult to remove, whereas the method based on the frequency domain can filter the noise with specific frequency, but has limited effect on the random noise. Therefore, the single domain method has a limitation in coping with complex mixed noise, and cannot achieve overall noise suppression. Furthermore, fine structure and texture detail preservation during noise reduction is also a technical challenge. The existing noise reduction algorithm is difficult to distinguish noise from high-frequency detail signals such as edges, textures and the like, excessive smoothing and key information loss are often caused, and the noise reduction effect and detail retention are difficult to balance. Even if the processing methods of different domains are combined, the prior art lacks an effective collaborative optimization mechanism. Most of the processing modules are independently designed, and lack of deep feature interaction and information complementation leads to the fact that the overall performance is limited by simple superposition of all parts, and a unified and efficient noise reduction system with complementary features is difficult to form. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a night vision image noise reduction method based on a double-domain joint model, which solves the technical problems that the prior art is difficult to effectively inhibit mixed noise and retain image detail textures at the same time when processing a night vision image. In order to solve the technical problems, the invention provides a night vision image noise reduction method based on a dual-domain joint model, which comprises the following steps: S1, acquiring an original Bayer domain image to be processed; S2, inputting the original Bayer domain image into a preset time domain noise reduction model to obtain a time domain noise reduction image; s3, inputting the original Bayer domain image into a preset frequency domain noise reduction model to obtain a frequency domain noise reduction image; s4, carrying out fusion processing on the time domain noise reduction image and the frequency domain noise reduction image to generate a target noise reduction image. In a specific embodiment, before the original Bayer domain image is obtained in step S1, the method further includes a step of calibrating an imaging system, where the calibration includes black level calibration, noise characteristic calibration, static dead point calibration, lens dark angle calibration, white balance coefficient calibration, and color correction matrix calibration. Preferably, in step S2, the time domain noise reduction model is a model based on a convolutional neural network, and processes the original Bayer domain image in a spatial dimension to obtain the time domain noise reduction image. Preferably, in step S3, the specific implementation manner of the method is that the frequency domain noise reduction model performs fourier transform on the original Bayer domain image to obtain a frequency domain representation of the original Bayer domain image, performs a filtering operation of multiplying the frequency domain representation element by learning a frequency domain filtering mask, and performs inverse fourier transform on the frequency domain representation after the filtering operation to obtain the frequency domain noise reduction image. This process can be represented by the following formula: ; Wherein: representing a frequency domain noise reduction image; representing an inverse fourier transform operation; representing network parameters generated by frequency domain noise reduction model learning And frequency variationA determined frequency domain filter mask; Representing the Hadamard product operation, i.e., element-wise multiplication; representing a fourier transform operation; Representing the original Bayer domain image. Preferably, in step S4, the specific implementation manner of the method is that attention weights are dynamically calculated according to feature maps of the time domain noise reduction model and the frequency do