Search

CN-120912473-B - Wavelet Mamba weak light image enhancement method based on illumination priori

CN120912473BCN 120912473 BCN120912473 BCN 120912473BCN-120912473-B

Abstract

The invention discloses a wavelet Mamba weak light image enhancement method based on illumination priori, which comprises the following specific steps of step 1, constructing an illumination estimation module, taking a weak light image as input to obtain an illumination quaternary priori Step 2, constructing a wavelet Mamba coding and decoding network, and quaternary priori comparing the weak light image with illumination The method comprises the steps of inputting the images to a wavelet Mamba coding and decoding network to obtain enhanced images, training an overall network composed of an illumination estimation module and the wavelet Mamba coding and decoding network to obtain a trained overall network, and inputting the weak light images to be enhanced to the trained overall network to obtain final enhanced images. The method of the invention can enhance the image while keeping naturalness and detail, and the enhanced image has high quality.

Inventors

  • ZHOU XIAOJING
  • Dong Jierui

Assignees

  • 上海沐钠信息科技有限公司

Dates

Publication Date
20260508
Application Date
20250825

Claims (6)

  1. 1. The method for enhancing the weak light image of the wavelet Mamba based on the illumination priori is characterized by comprising the following specific steps: step 1, constructing an illumination estimation module, and taking a weak light image as input to obtain an illumination quaternary prior ; In step 1, the specific processing procedure of the illumination estimation module is as follows: Step 1.1, the input dim light image passes through a learnable linear mapping layer Shallow features are obtained and the input low-light image is sequentially passed through a 1×1 convolution and gaussian filter Processing to obtain enhanced features; Step 1.2, calculating the spectral intensity of the shallow layer characteristics obtained in the step 1.1 Spectral slope And spectral curvature ; Wherein, the Is the pixel spatial location of the shallow feature, X represents a horizontal direction coordinate, and y represents a vertical direction coordinate; Representing wavelength; step 1.3, the spectrum intensity obtained in the step 1.2 is used Spectral slope And spectral curvature Respectively multiplying the obtained enhanced features obtained in the step 1.1 element by element to obtain weighted spectrum intensity Spectral slope And spectral curvature ; Step 1.4, calculating a tone prior Channel prior Texture prior Color prior ; In step 1.4, hue prior The expression of (2) is: (5); in the formula, And Are all learnable parameters; Channel prior The expression of (2) is: (6); in the formula, Indicating the reflectivity of the material Is a gaussian filtered first order spectral derivative of (a); Indicating the reflectivity of the material Is a gaussian filtered second-order spectral derivative of (c); The expression of texture a priori W is: (7) (8); in the formula, Is the reflectivity of the material Is a function of the laplace operator of (c), Set to 0.01 to prevent zero removal; Color prior The expression of (2) is: (9) in the formula, For the normalized red channel component, For the normalized green channel component, Is a normalized blue channel component; 、 、 by combining the spectral intensities obtained in step 1.3 Respectively performing linear normalization processing on RGB color channels of (C) to be processed The pixel value at the position is mapped to the interval [ -1,1] to obtain; step 1.5, the color tone obtained in step 1.4 is used for priori Channel prior Texture prior And color prior Splicing in the channel dimension to form an illumination quaternary prior; step2, constructing a wavelet Mamba coding and decoding network, and quaternary priori relating the dim light image and illumination Inputting the images to a wavelet Mamba coding and decoding network to obtain enhanced images; in step 2, the wavelet Mamba codec network includes a first convolution layer, an encoder, an illumination prior fusion module, a decoder, and an output convolution layer; the specific processing procedure of the wavelet Mamba coding and decoding network is as follows: Step 2.1, the H multiplied by W multiplied by C weak light image is input into a first convolution layer to extract initial characteristics, and the initial characteristics are input into an encoder to obtain A feature map; step 2.2, the step 2.1 Feature map and illumination quaternary prior Inputting the illumination prior fusion characteristics to an illumination prior fusion module to obtain the illumination prior fusion characteristics ; The processing procedure of the illumination priori fusion module is as follows: The mixture obtained in the step 2.1 Sequentially performing layer normalization and Fourier transformation operations on the feature map to obtain a Query vector, and performing the light quaternary priori obtained in the step 1 Sequentially performing layer normalization and Fourier transformation operations to obtain Key vectors, and performing the illumination quaternary priori obtained in the step 1 Sequentially performing layer normalization and convolution operation to obtain Value, obtaining hidden space features through Query vectors, key vectors and Value values, and performing linear transformation on the hidden space features and the hidden space features Adding the feature graphs to obtain the illumination priori fusion feature ; Step 2.3, fusing the illumination priori features obtained in the step 2.2 Sequentially inputting the output of the third wavelet Mamba module in the step 2.1 into a decoder and an output convolution layer for processing to obtain an enhanced image; Step 3, training an overall network consisting of an illumination estimation module and a wavelet Mamba coding and decoding network to obtain a trained overall network; and 4, inputting the weak light image to be enhanced into a trained whole network to obtain a final enhanced image.
  2. 2. The illumination prior based wavelet Mamba low-light image enhancement method according to claim 1, wherein in step 1.2, the spectral intensity of the shallow features is The expression of (2) is: (1) in the formula, For the spectrum of the light source, For the purpose of specular reflection, Is the reflectivity of the material; assuming equal energy illumination, the spectrum of the light source Simplified to and wavelength Independent of each other The following steps are: (2) spectral slope of shallow features And spectral curvature of shallow features The expressions are respectively: (3) (4)。
  3. 3. the illumination prior based wavelet Mamba low-light image enhancement method according to claim 1, wherein in step 2.1, the convolution kernel of the first convolution layer is 3 x 3; The encoder sequentially comprises a first wavelet Mamba module, a first downsampling module, a second wavelet Mamba module, a second downsampling module, a third wavelet Mamba module and a third downsampling module; The processing procedure of each wavelet Mamba module is as follows: normalizing input features, carrying out linear transformation through a data_transform function, mapping to an [ -1,1] interval to obtain mapped features, and decomposing the mapped features into a low-frequency component and a high-frequency component through Haar wavelet transformation; Sequentially performing 3×3 convolution and ReLU activation operation on the low-frequency component to obtain a first processed feature, sequentially passing through a second convolution layer, a ReLU activation function and a third convolution layer to obtain the first convolved feature, remolding the first convolved feature into [ batch_size, sequence_length, d_model ] according to a channel priority mode, wherein the remolded sequence_length is obtained by remolding the channel number, the remolded d d_model is the product of the height and the width before remolding, the remolded feature is input into a first state space model to be processed to obtain a first updated feature, the state space model uses mamba _ ssm in a third party library Mamba and scans along the sequence length, and sequentially performing 3×3 convolution sum layer normalization operation on the first updated feature to obtain the enhanced low-frequency component; sequentially performing 3×3 convolution and ReLU activation operations on the high-frequency component to obtain a second processed feature, sequentially passing through a fourth convolution layer, a ReLU activation function and a fifth convolution layer to obtain a second convolved feature, and sequentially performing 3×3 convolution sum layer normalization operations on the second convolved feature according to a space dimension priority mode to obtain an enhanced high-frequency component, wherein the remodeled sequence length is the product of the height and the width before remodelling, the remodeled d model is obtained by remodelling the channel number, the remodeled feature is input into a second state space model to be processed to obtain a second updated feature, and the second state space model uses mamba _ ssm in a third party library Mamba to sequentially perform 3×3 convolution sum layer normalization operations on the second updated feature; Performing inverse discrete wavelet transform on the obtained enhanced low-frequency component and enhanced high-frequency component to obtain frequency enhanced hidden characteristics 。
  4. 4. The illumination prior based wavelet Mamba low-light image enhancement method according to claim 1, wherein in step 2.2, The expressions of the Query vector, the Key vector, the Value and the hidden space feature are as follows: (5) Wherein Q represents a Query vector, K represents a Key vector, V represents a Value, FFT represents Fourier transform, and SoftMax represents a SoftMax activation function; 、 And Is a learnable unbiased linear projection parameter.
  5. 5. The illumination prior based wavelet Mamba low-light image enhancement method according to claim 1, wherein in step 2.3, the decoder is sequentially composed of a first upsampling, a fourth wavelet Mamba module, a second upsampling, a fifth wavelet Mamba module, a third upsampling, and a sixth wavelet Mamba module; The convolution kernel of the output convolution layer is 3×3; the specific process is as follows: illumination prior fusion feature As the first up-sampled input, the output of the third wavelet Mamba module is added to the first up-sampled output to be used as the input of the fourth wavelet Mamba module, the output of the fourth wavelet Mamba module is used as the second up-sampled input, the output of the second wavelet Mamba module is added to the second up-sampled output to be used as the input of the fifth wavelet Mamba module, the output of the fifth wavelet Mamba module is used as the third up-sampled input, the output of the first wavelet Mamba module is added to the third up-sampled output to be used as the input of the sixth wavelet Mamba module, and the output of the sixth wavelet Mamba module is input into the output convolution layer to obtain the enhanced image.
  6. 6. The illumination prior-based wavelet Mamba low-light image enhancement method according to claim 1, wherein the total loss function in the training process is: (10) in the formula, I=1, 2, 3, 4 for the super parameter; Loss value for L-1; loss for reconstruction; Is a perceived loss; is a smoothing loss; Wherein, the (11) In the formula (11), the amino acid sequence of the compound, Is the low frequency component of the input image decomposed by Haar wavelet transform, Is a low frequency component of real data; (12) in the formula, For the pixel coordinates of the image space, In order to enhance the image is, Is a true value image; (13) in the formula, Representing a VGG network pre-trained with an ImageNet dataset; Features of the truth image extracted for the VGG network, Features of the enhanced image extracted for the VGG network; (14) in the formula, And Respectively shown in And The gradient of the direction is such that, Representing the enhanced image.

Description

Wavelet Mamba weak light image enhancement method based on illumination priori Technical Field The invention belongs to the technical field of image enhancement methods, and particularly relates to a wavelet Mamba weak light image enhancement method based on illumination priori. Background Dim light enhancement techniques have found wide application in various fields including visual monitoring, autopilot and computed radiography. In particular, smart phone photographing has become ubiquitous and very prominent. Camera photographing using smartphones in dim environments is particularly challenging, limited by camera aperture size, real-time processing requirements, and memory limitations. In recent years, with the vigorous development of deep learning technology, a low-light image enhancement method based on deep learning is rapidly developed. The deep learning method can be self-adaptive to different scenes and illumination conditions, effectively processes complex and changeable actual image data, provides more accurate and comprehensive illumination priori information for subsequent image enhancement processing, greatly improves the image enhancement effect and adaptability, and expands the application range and practical value of the weak light image enhancement technology. The learning-based model comprises a CNN-based method and a Transformer-based method, and is mainly formed by constructing a Convolutional Neural Network (CNN) or a variant network, and utilizing a large number of low-light images and corresponding normal illumination images thereof as training data, so that the network automatically learns mapping relations between the low-light images and the normal illumination images, thereby generating more clear, natural and detail-rich enhanced images. However, obtaining paired training data is often difficult and heavy in practical applications, which limits the generalization ability of the model to some extent. Although the deep learning-based method achieves remarkable results in the aspect of weak light image enhancement, some problems and challenges to be solved still exist, the deep learning-based method lacks long-distance dependence, so that illumination distribution is balanced, overexposure and underexposure occur on an enhanced image, image quality is affected, most of the deep learning-based methods process in an original space, mainly focus on local pixel values and texture information, are difficult to capture global structure and semantic information, have limited distinguishing and processing capacities on different areas in a complex scene, are sensitive to noise, are easy to amplify noise in the enhancement process, and affect image quality. Accordingly, there remains a need in the art for a low-light image enhancement method that can improve global enhancement, reduce computing resource consumption, and at the same time effectively suppress noise. Disclosure of Invention The invention aims to provide a wavelet Mamba weak light image enhancement method based on illumination priori, which solves the problem of poor enhanced image quality of the existing method. The technical scheme adopted by the invention is that the method for enhancing the weak light image of the wavelet Mamba based on illumination priori comprises the following specific steps: Step 1, constructing an illumination estimation module, and taking a weak light image as input to obtain an illumination quaternary prior ; Step2, constructing a wavelet Mamba coding and decoding network, and quaternary priori relating the dim light image and illuminationInputting the images to a wavelet Mamba coding and decoding network to obtain enhanced images; Step 3, training an overall network consisting of an illumination estimation module and a wavelet Mamba coding and decoding network to obtain a trained overall network; and 4, inputting the weak light image to be enhanced into a trained whole network to obtain a final enhanced image. The invention is also characterized in that: In step 1, the specific processing procedure of the illumination estimation module is as follows: Step 1.1, the input dim light image passes through a learnable linear mapping layer Shallow features are obtained and the input low-light image is sequentially passed through a 1×1 convolution and gaussian filterProcessing to obtain enhanced features; Step 1.2, calculating the spectral intensity of the shallow layer characteristics obtained in the step 1.1 Spectral slopeAnd spectral curvature; Step 1.3, the spectrum intensity obtained in the step 1.2 is usedSpectral slopeAnd spectral curvatureRespectively multiplying the obtained enhanced features obtained in the step 1.1 element by element to obtain weighted spectrum intensitySpectral slopeAnd spectral curvature; Step 1.4, calculating a tone priorChannel priorTexture priorColor prior; Step 1.5, the color tone obtained in step 1.4 is used for prioriChannel priorTexture priorAnd color priorAnd splicing i