Search

CN-120634898-B - Night image halation removing method

CN120634898BCN 120634898 BCN120634898 BCN 120634898BCN-120634898-B

Abstract

The invention provides a night image halation removing method which comprises the steps of constructing priori knowledge, constructing an initialization network and a depth expansion network according to the priori knowledge, extracting a variable initial value set from a night halation polluted image by adopting the initialization network, inputting the night halation polluted image and the variable initial value set into the depth expansion network for multiple iterative operation, and extracting a final halation removing image from the night halation polluted image, wherein the depth expansion network comprises a plurality of near-end networks, and each near-end network participates in one iterative operation. Under the condition of fully utilizing priori knowledge, the invention adopts the depth expansion network with a plurality of near-end networks to iterate the mapping image, the halation-free feature image, the constraint variable and the halation removal image, thereby extracting the final halation removal image, better retaining image textures and restoring image details, and better removing facula and streak artifacts in the image on the premise of not increasing the application cost.

Inventors

  • LIU YUN
  • YANG GUANG

Assignees

  • 西南大学

Dates

Publication Date
20260508
Application Date
20250615

Claims (7)

  1. 1. A night image halation removal method, comprising: Constructing priori knowledge, wherein the priori knowledge comprises a pixel level space consistency term, a feature level detail enhancement term and a feature level noise suppression term; Constructing an initialization network and a depth expansion network according to priori knowledge, and extracting a variable initial value set from the night halation pollution image by adopting the initialization network, wherein the variable initial value set comprises an initial halation mask image, an initial halation removal image, an initial mapping image, an initial halation-free feature image and an initial constraint variable; inputting the night halation pollution image and the variable initial value set into a depth expansion network for multiple iterative operation, and extracting a final halation removal image from the night halation pollution image, wherein the depth expansion network comprises a plurality of near-end networks, and each near-end network participates in one iterative operation; the constructing a priori knowledge includes: the halo layer decomposition model is constructed according to the following formula: ; wherein I is a night halation pollution image, B is a halation removal image, and F is a halation mask image; constructing a first optimization function corresponding to the pixel-level space consistency term: ; wherein f represents the fibonacci norm, The method comprises the steps that a fibonacci norm item is used for guaranteeing the fidelity of image data, and E 1 is a first optimization function; the constructing a priori knowledge includes: extracting a feature map by using a convolution dictionary according to the following formula: ; ; ; wherein ZI is a characteristic diagram of a night halation pollution image, ZB is a halation-free characteristic diagram, ZF is a halation mask characteristic diagram, D is a convolution dictionary, Is a convolution operator; Constructing a second optimization function containing the feature level detail enhancement terms: ; where M is a map, x is an element-wise multiplication, As a result of the first penalty factor, For the second penalty coefficient, For the first regularization coefficient, E 2 is a second optimization function for the first prior term; the constructing a priori knowledge includes: constructing a feature level noise prior term according to the following formula: ; Wherein, the For the second regularization coefficient, A norm obtained by multiplying the non-halation characteristic diagram and the halation mask characteristic diagram element by element; Constructing a third optimization function of the feature level noise suppression term: ; Wherein, the For the convolution of the convolution dictionary and the halo mask image, E 3 is a third optimization function, which is a norm of the element-wise product of the no halo feature map.
  2. 2. The method for removing night image halation according to claim 1, further comprising, before the step of extracting the variable initial value set from the night halation contaminated image using the initializing network: Calculating Charbonnier consistency loss between the halo removal image and the true value to obtain a first loss function; calculating Charbonnier consistency loss between the halation mask image and the training set to obtain a second loss function; Adding the first loss function and the second loss function to obtain a third loss function; calculating the VGg consistency loss between the halo removal image and the true value to obtain a fourth loss function; calculating the VGg consistency loss between the halation mask image and the training set to obtain a fifth loss function; Adding the fourth loss function and the fifth loss function to obtain a sixth loss function; Carrying out weighted summation on the third loss function and the sixth loss function to obtain a final loss function; And adopting the final loss function to perform end-to-end training on the initialization network and the deep expansion network simultaneously.
  3. 3. The night image halation removal method according to claim 1, wherein the inputting the night halation contaminated image and the set of variable initial values into a depth expansion network for a plurality of iterative operations comprises: When the iteration operation is carried out for the t time, calculating the gradient of the t mapping image, inputting the gradient of the t mapping image and the gradient of the t mapping image into the t near-end network of the depth expansion network, and calculating the t+1 mapping image; Computing the gradient of the t no-halo feature map, inputting the t no-halo feature map, the gradient of the t no-halo feature map and the t no-halo mask feature into a soft threshold operator to obtain the t+1th no-halo feature map; calculating the gradient of the t constraint variable, inputting the gradient of the t constraint variable and the gradient of the t constraint variable into a t near-end network of a deep expansion network, and calculating the t+1th constraint variable; The gradient of the t-th halo removal image is calculated, and the t+1th halo removal image is calculated according to the t-th halo image and the gradient of the t-th halo removal image.
  4. 4. A night image halation removal method according to claim 3, wherein the inputting the gradients of the t-th map and the t-th map into the t-th near-end network of the depth expansion network, calculating the t+1-th map comprises: the t+1st map is calculated according to the following formula: ; Wherein M t+1 is the t+1th map, M t is the t-th map, For the gradient of the t-th map, The parameters are updated for the map and, For element-wise multiplication prox t1 is the t1 st near-end network of the map of the deep-expanded network.
  5. 5. A night image halo removal method according to claim 3, wherein said inputting the t-th no-halo feature map, the gradient of the t-th no-halo feature map, and the t-th halo mask feature into a soft threshold operator, results in the t+1-th no-halo feature map, comprising: The t+1st halation-free feature map is calculated according to the following formula: ; Wherein ZB t+1 is the t+1th halation-free feature map, soft is the soft threshold operator, ZB t is the t halation-free feature map, To update the parameters for the no halation feature map, For the t-th gradient without halation profile, For the second regularization coefficient, As a result of the first penalty factor, For the second penalty factor, D t is the t-th convolution dictionary, In the form of a halo mask image, Is a convolution operator, which is an element-by-element multiplication.
  6. 6. A night image halo removal method according to claim 3, wherein said calculating the t+1th constraint variable comprises: The t+1th constraint variable is calculated according to the following formula: ; Wherein N t+1 is the t+1th constraint variable, prox t2 is the t2 nd proximal network of constraint variables of the deep expansion network, N t is the t constraint variable, In order to constrain the updated parameters of the variables, Gradient for the t constraint variable, element-wise multiplication.
  7. 7. A night image halo removal method according to claim 3, wherein said calculating a t+1th halo removal image comprises: the t+1th halo removal image is calculated according to the following formula: ; wherein B t+1 is the t+1th halo removal image, B t is the t th halo removal image, The parameters are updated for the halo removal image, The gradient of the image is removed for the t-th halo, element-wise multiplication.

Description

Night image halation removing method Technical Field The invention relates to the technical field of image processing, in particular to a night image halation removing method. Background When shooting is performed at night, strong light scattering or reflection phenomenon occurs in the lens, light spots and streak artifacts can be generated, so that contrast and detail definition of an image can be reduced, and visual effect and performance of an image processing algorithm are affected. For example, a stereoscopic camera in a night driving scene may erroneously recognize a halation as an obstacle. In the scenario where an unmanned aerial vehicle tracks an aerial target, an erroneous target may be tracked. The first type of method of reducing halation is to optimize the hardware design, e.g., using a special lens or coating the lens with an anti-reflective coating, etc., but these hardware improvements do not adequately remove halation while increasing the cost of application and also contaminating the lens with unpredictable spots. The second method for eliminating halation is to design an image processing algorithm to remove halation in an image, and recently a learning-based method has a certain effect in the field of image restoration, such as selecting a plurality of image restoration models as a base line model, introducing fast Fourier transform to extract global frequency characteristics according to a Swin transducer, combining a fine tuning strategy by means of a cascade neural network, and simultaneously constructing triplets and contrast learning to optimize the models. Therefore, a method is needed that can fully utilize prior information to better preserve image texture and restore image details, and better remove light spots and streak artifacts in images without increasing application cost. Disclosure of Invention In order to overcome the problems in the related art, the invention aims to provide a night image halation removing method which can fully utilize prior information to better reserve image textures and restore image details and better remove light spots and streak artifacts in images on the premise of not increasing the application cost. A night image halo removal method comprising: Constructing priori knowledge, wherein the priori knowledge comprises a pixel level space consistency term, a feature level detail enhancement term and a feature level noise suppression term; Constructing an initialization network and a depth expansion network according to priori knowledge, and extracting a variable initial value set from the night halation pollution image by adopting the initialization network, wherein the variable initial value set comprises an initial halation mask image, an initial halation removal image, an initial mapping image, an initial halation-free feature image and an initial constraint variable; Inputting the night halation pollution image and the variable initial value set into a depth expansion network for multiple iterative operation, and extracting a final halation removal image from the night halation pollution image, wherein the depth expansion network comprises a plurality of near-end networks, and each near-end network participates in one iterative operation. In a preferred technical solution of the present invention, the constructing priori knowledge includes: the halo layer decomposition model is constructed according to the following formula: ; wherein I is a night halation pollution image, B is a halation removal image, and F is a halation mask image; constructing a first optimization function corresponding to the pixel-level space consistency term: ; wherein f represents the fibonacci norm, The Fiboner-contract norm term is used for guaranteeing the fidelity of the image data, and E 1 is a first optimization function. In a preferred technical solution of the present invention, the constructing priori knowledge includes: extracting a feature map by using a convolution dictionary according to the following formula: ; ; ; wherein ZI is a characteristic diagram of a night halation pollution image, ZB is a halation-free characteristic diagram, ZF is a halation mask characteristic diagram, D is a convolution dictionary, Is a convolution operator; Constructing a second optimization function containing the feature level detail enhancement terms: ; where M is a map, x is an element-wise multiplication, As a result of the first penalty factor,For the second penalty coefficient,For the first regularization coefficient,E 2 is a second optimization function for the first prior term. In a preferred technical solution of the present invention, the constructing priori knowledge includes: constructing a feature level noise prior term according to the following formula: ; Wherein, the For the second regularization coefficient,A norm obtained by multiplying the non-halation characteristic diagram and the halation mask characteristic diagram element by element; Constructing a thir