Search

CN-120147184-B - Image defogging algorithm based on multi-scale pulse neural network

CN120147184BCN 120147184 BCN120147184 BCN 120147184BCN-120147184-B

Abstract

The invention provides an image defogging algorithm based on a multi-scale pulse neural network. The method comprises the steps of preprocessing an input fog image, extracting initial features through two-dimensional convolution, constructing a base layer formed by a multi-scale LIF (Leaky Integrate andFire) module, acquiring multi-scale features by using convolution kernels of different scales, simulating a pulse issuing mechanism of biological neurons to extract the features, carrying out feature fusion through SK-fusion after full-connection layer mapping, and finally inputting the two-dimensional convolution to restore an embedded feature image to obtain a clear defogging image. The method combines the high efficiency of the impulse neural network and the accuracy of the full-connection neural network, remarkably improves the definition and detail recovery effect of defogging images, and is suitable for various image processing scenes.

Inventors

  • LU YANMING
  • LIU MINGZHE

Assignees

  • 成都理工大学

Dates

Publication Date
20260508
Application Date
20250226

Claims (8)

  1. 1. An image defogging method based on a multi-scale impulse neural network, comprising: preprocessing an input fog image, wherein the preprocessing comprises image normalization and data enhancement; executing an embedding program for extracting preliminary features on the preprocessed image; Constructing a base layer consisting of multi-scale LIF, wherein the base layer comprises multi-scale convolution, LIF pulse distribution and full connection, the full connection consists of two-dimensional convolution connected by LeakyReLU, the construction of the base layer consisting of multi-scale LIF comprises the steps of extracting image features based on the convolution layer, carrying out channel mixing operation by utilizing the image features, simulating a pulse distribution mechanism of biological neurons based on the LIF neuron layer, carrying out token mixing operation, converting the output of the LIF neuron layer into pulse signals based on an activation function layer, carrying out fusion on the characteristics of an input characteristic map and an original image based on a linear transformation layer, forming an output port of a neural network based on the activation function layer, incorporating the linear transformation layer and the activation function layer into the full connection neural network layer, and connecting each full connection neural network layer to form full connection; Inputting the image into a pulsing structure of the LIF to form a pulsing mechanism that mimics a biological neuron; Carrying out drop path regularization operation on the LIF Module output image; inputting an image to a full-connection layer, and performing regularization treatment; Performing feature fusion on the image by using SK-fusion; Returning to the embedding program for extracting the preliminary features of the preprocessed image until the number of circulation times is a numerical value N, wherein N is a positive integer; canceling embedding of the output image; And restoring the embedded characteristic diagram to realize defogging of the image.
  2. 2. The method for image defogging based on the multi-scale pulse neural network of claim 1, wherein the preprocessing of the input fog image comprises the following steps: The data is scaled such that its pixel values fall within a range between [ -1, 1 ].
  3. 3. The method of image defogging based on a multi-scale impulse neural network of claim 2, wherein said inputting the image into the impulse firing structure of the LIF to form an impulse firing mechanism simulating biological neurons comprises: forming an MLP neural network layer based on the LIF; converting the input features into embedded vectors using an MLP neural network layer; Based on the embedded vector, input parameters for the impulse neuron module in the impulse delivery structure are formed.
  4. 4. The method for image defogging based on a multi-scale impulse neural network of claim 3, wherein forming an MLP neural network layer based on LIF comprises: adding a dwconv to the first MLP layer; AxialShift to replace the MLP layer based on LIF; Accumulating and propagating information in the horizontal direction of the image features based on the horizontal LIF of the MLP layer; information is accumulated and propagated in the vertical direction of the image features based on the vertical LIF of the MLP layer.
  5. 5. The method of image defogging based on a multi-scale impulse neural network of claim 4, wherein said inputting the image into the impulse firing structure of the LIF to form an impulse firing mechanism simulating biological neurons, further comprises: The pulse issuing mechanism of the biological neuron is simulated by using the information accumulated and propagated in the horizontal direction of the image features and the information accumulated and propagated in the vertical direction of the image features so as to perform feature mixing.
  6. 6. The image defogging method based on the multi-scale impulse neural network of claim 3, wherein the image defogging algorithm based on the multi-scale impulse neural network further comprises the behavior modeling of LIF model: When (when) At the time of membrane potential Decaying with time and receiving external input : When the membrane potential Reaching a threshold value At the time of neuron firing, the membrane potential is reset to : , Wherein the method comprises the steps of Is the membrane potential, the membrane is a membrane, Is an input from an upper layer of the system, Is a coefficient of time that is a function of the time, Is an output of the device and is, Is the firing threshold of this neuron, and when a pulse is triggered, the membrane potential u is reset to Where t represents the index of the packet.
  7. 7. The image defogging method based on the multi-scale impulse neural network of claim 6, wherein the image defogging algorithm based on the multi-scale impulse neural network further comprises full-precision LIF modeling: The output 1 is replaced by u, the original binarized output is changed into linear output, so that full-precision information is reserved, and a full-precision LIF model is applied to iterative LIF to obtain: Transpose of the weight matrix, Is an input feature, r t+1 n is the final full-precision output of the t+1 step, o t+1 n is a temporary variable recording the output state of the t+1 step, and the coefficient And Parameters are learned for the neural network.
  8. 8. The method for image defogging based on a multi-scale impulse neural network of claim 7, wherein said image defogging based on a multi-scale impulse neural network further comprises a back propagation model modeling of explicit iterative LIF neuron chain law: where L is the dynamic range of the pixel value.

Description

Image defogging algorithm based on multi-scale pulse neural network Technical Field The application relates to the field of image processing, in particular to an image defogging algorithm based on a multi-scale pulse neural network. Background Due to the influence of factors such as fog, haze, atmospheric pollution and the like, the image detail and contrast can be weakened, and the color can be distorted. Therefore, image defogging and restoration appears to be extremely important. The image enhancement-based method, the physical model-based method and the deep learning-based method are conventional image processing methods. The method based on the physical model is to start from an atmospheric scattering model, establish a mathematical model of an image degradation process, and then perform inversion reduction on a clear image. The best known of such methods is the dark channel Prior algorithm (DARK CHANNEL primary, DCP). The DCP algorithm is based on an important observation that in a natural scene, in one or more color channels of most non-sky areas, at least one pixel with very low pixel value is present. With this a priori knowledge, the DCP can effectively estimate the haze image, thereby restoring a clear image. Although DCP performs well in many situations, it is highly complex and does not work well in some scenarios. Therefore, a defect of poor comprehensive use effect of the conventional image processing method is necessary, and an image defogging algorithm based on a multi-scale pulse neural network is provided. Disclosure of Invention The summary of the application is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. The summary of the application is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. In order to solve the technical problems mentioned in the background section above, some embodiments of the present application provide an image defogging algorithm based on a multi-scale impulse neural network, comprising: preprocessing an input fog image, wherein the preprocessing comprises image normalization and data enhancement; executing an embedding program for extracting preliminary features on the preprocessed image; constructing a base layer consisting of multi-scale LIF, wherein the base layer comprises multi-scale convolution, LIF pulse emission and full connection, and the full connection consists of LeakyReLU connected two-dimensional convolutions; Inputting the image into a pulsing structure of the LIF to form a pulsing mechanism that mimics a biological neuron; Carrying out drop path regularization operation on the LIF Module output image; inputting an image to a full-connection layer, and performing regularization treatment; Performing feature fusion on the image by using SK-fusion; Returning to the embedding program for extracting the preliminary features of the preprocessed image until the number of circulation times is a numerical value N, wherein N is a positive integer; canceling embedding of the output image; And restoring the embedded characteristic diagram to realize defogging of the image. Further, the data is scaled such that its pixel values fall within a range between [ -1,1 ]. Further, extracting image features based on the convolution layer; channel mixing work is carried out by utilizing image characteristics; simulating a pulse issuing mechanism of the biological neurons based on the LIF neuron layer; Performing token mixing; Based on the activation function layer, the output of the LIF neuron layer is converted into a pulse signal. Further, based on the linear transformation layer, the input feature map and the features of the original image are fused; Forming an output port of the neural network based on the activation function layer; Incorporating the linear transformation layer and the activation function layer into a fully connected neural network layer; each fully-connected neural network layer is connected to form a fully-connected. Further, forming an MLP neural network layer based on the LIF; converting the input features into embedded vectors using an MLP neural network layer; Based on the embedded vector, input parameters for the impulse neuron module in the impulse delivery structure are formed. Further, adding a dwconv to the first MLP layer; AxialShift to replace the MLP layer based on LIF; Accumulating and propagating information in the horizontal direction of the image features based on the horizontal LIF of the MLP layer; information is accumulated and propagated in the vertical direction of the image features based on the vertical LIF of the MLP layer. Further, the pulse issuing mechanism of the biological neuron is simulated by using the information accumulated and propagated in the horizontal direction of the image features and the information accumu