CN-121526927-B - FPGA-based video real-time defogging method

CN121526927BCN 121526927 BCN121526927 BCN 121526927BCN-121526927-B

Abstract

The invention discloses a video real-time defogging method based on an FPGA (field programmable gate array), which is implemented by utilizing a video stream inter-frame similarity design pipeline architecture, and comprises the steps of dividing a sky area and a non-sky area of a foggy image by using a sky area division algorithm integrating a minimum channel value and a four-direction Sobel operator gradient value, taking the average value of the minimum channel value of the sky area as an atmospheric light value channel component, calculating to obtain an atmospheric light value, calculating the initial transmittance of the whole foggy image, not processing the initial transmittance of the non-sky area, correcting the initial transmittance of the sky area by adopting a transmittance self-adaptive compensation mechanism designed based on the difference value of the minimum channel average value and the minimum channel value of a single pixel, and recovering the foggy image into a foggy image based on the atmospheric light value, the final transmittance and an atmospheric scattering model. The method can maintain low complexity while guaranteeing defogging performance, and is easy for FPGA deployment.

Inventors

WU TIANYA
YANG SICONG
YU WENJIE
LI WEI

Assignees

南昌大学

Dates

Publication Date: 20260508
Application Date: 20260116

Claims (4)

1. A video real-time defogging method based on an FPGA is characterized in that a pipeline architecture is designed by utilizing the similarity between frames of video streams, and the pipeline architecture adopts an inter-frame parameter multiplexing strategy to carry out nth Atmospheric light value calculated by 1 frame And a brightness threshold Directly transferring to the nth frame for use, and simultaneously transferring the atmospheric light value of the nth frame And a brightness threshold Transferring to the n+1st frame, and executing the following steps by adopting a processing flow of unidirectional time sequence propulsion: A sky area segmentation algorithm integrating the minimum channel value and the gradient value of the four-direction Sobel operator is used, brightness segmentation is realized by adopting the minimum channel value, gradient segmentation is realized by adopting the four-direction Sobel operator, all pixel points are traversed, brightness and gradient threshold segmentation is carried out, a sky area mask is obtained after binarization calculation and morphological filtering, and a sky area and a non-sky area of a foggy image are segmented; taking the average value of the minimum channel value of the sky area as the channel component of the atmosphere light value Then calculate the atmospheric light value ; First, the initial transmissivity of the whole foggy image is calculated The initial transmittance of the non-sky area is not processed, a transmittance self-adaptive compensation mechanism designed based on the difference value between the minimum channel mean value and the minimum channel value of a single pixel is adopted, and the initial transmittance of the sky area is corrected to obtain the final transmittance ; Based on atmospheric light values Final transmittance And an atmospheric scattering model for restoring the foggy image to a foggy image; The method comprises the steps of optimizing an initial transmissivity calculation formula and a compensation transmissivity calculation formula in an 8-bit quantization mode, converting floating point number operation into fixed point number multiplication operation, and reducing the consumption of FPGA hardware resources; The processing steps of the sky area segmentation algorithm specifically comprise: calculating a minimum channel diagram of the foggy image, and calculating the average value to obtain the minimum channel average value Multiplying by a coefficient Obtaining brightness threshold of sky region segmentation ; Carrying out gradient calculation on the minimum channel map by adopting a four-direction Sobel operator to obtain a four-direction Sobel operator gradient map, recording a gradient value of each pixel point, and setting a gradient threshold value of sky region segmentation ; Traversing the hazy image to make the minimum channel value larger than And gradient value is smaller than The pixel points marked as1 and the rest pixel points marked as 0, wherein the pixel points marked as1 form a sky area, and the pixel points marked as 0 form a non-sky area, so as to obtain a binary initial mask of the sky area; Morphological filtering is carried out on the binarized initial mask to obtain a final sky area mask; Wherein the atmospheric light value channel component The calculation formula of (2) is as follows: In the formula (5), the amino acid sequence of the compound, Refers to an RGB color channel of an image, and an atmospheric light value channel component of an RGB three-channel of a sky area Specifically comprises , Is the R-channel component of the atmospheric light value, Is the G-channel component of the atmospheric light value, Is the B channel component of the atmospheric light value and satisfies the relationship , Representing a segmented sky region pixel set, Is the total number of pixels in the sky region, For a single pixel Is a minimum channel value of (2); wherein the final transmittance The calculation formula of (2) is as follows: In the formula (7), the amino acid sequence of the compound, In order to compensate for the reference value, For a single pixel Is used to determine the channel value of the channel, As an atmospheric light value channel component, In order for the initial transmittance to be the same, Representing the segmented sky region pixel sets; The method further comprises the step of deforming the compensation transmissivity calculation formula by adopting an 8-bit quantization mode to obtain the following formula: in the formula (10), the amino acid sequence of the compound, In order to compensate for the transmittance of the light, In order to compensate for the reference value, For a single pixel Is used to determine the channel value of the channel, As an atmospheric light value channel component, Is the initial transmittance.
2. The method of claim 1, wherein gradient values are calculated using a four-way Sobel operator The formula of (2) is as follows: in the formula, 、、、 The gradient change values obtained after convolution kernels in the four directions of horizontal, vertical, 45 degrees and 135 degrees are convolved with the image, For maximum value.
3. The method of claim 1, further comprising warping the initial transmittance calculation formula using an 8-bit quantization and based on the atmospheric light value channel components of the RGB three-channel system After simplification of the characteristics of consistent size, the following formula is obtained: in the formula (9), the amino acid sequence of the compound, In order for the initial transmittance to be the same, To prevent excessive defogging of the mist retention factor, As a foggy image Middle pixel Is used for the dark channel value of (1), Is the atmospheric light value channel component.
4. The method of claim 1, wherein the haze-free image is calculated as follows: In the formula (11), the amino acid sequence of the compound, In the form of a haze-free image, In the case of a hazy image, In the case of an atmospheric light value, In order to achieve a final transmittance of the light, A lower transmittance limit for preventing the transmittance from being too low.

Description

FPGA-based video real-time defogging method Technical Field The invention relates to the technical Field of digital image processing, in particular to a video real-time defogging method based on Field Programmable Gate Array (FPGA) GATE ARRAY. Background In haze weather, the outdoor visual sensors such as intelligent automobile data recorder and traffic monitoring cameras can have fuzzy and detail loss phenomena due to atmospheric scattering effect on collected video pictures, so that the sensing precision of embedded application scenes such as automatic driving and intelligent traffic monitoring is greatly reduced, the follow-up target recognition precision and management work efficiency are seriously influenced, and serious challenges are brought to intelligent traffic. In order to improve the definition of the video, an image defogging algorithm is required to process the video. Currently, mainstream defogging algorithms can be classified into three categories, namely image enhancement-based, physical model-based and deep learning-based. The algorithm based on the physical model is a preferable scheme of the embedded defogging scene because of moderate calculation complexity and stable defogging effect. The algorithm relies on an atmospheric scattering model, and has the core task of accurately estimating the transmissivity and the global atmospheric light value, so as to restore the haze-free image. The embedded defogging scene has strict requirements on real-time performance and low power consumption, and the FPGA is used as a programmable logic circuit device, has the characteristics of low power consumption, strong data parallel processing capability and low delay, and becomes an ideal choice for hardware implementation in the scene. Among various algorithms based on physical models, the dark channel prior defogging algorithm is a common algorithm for realizing real-time defogging by using an FPGA at present because of simple principle, low complexity and good defogging effect. In the process of realizing the technical scheme of the invention, at least the following technical problems in the prior art are found: Firstly, a dark channel priori defogging algorithm based on a physical model is realized by using an FPGA, and when a foggy image with a sky area is processed, the sky area does not accord with the dark channel priori principle, so that the obtained transmissivity is smaller, and further, the problem of large-area color distortion in the subsequent restoration process is caused, and the defogging effect is poor. The existing methods such as guide filtering optimize defogging effect by refining transmissivity, and the guide filtering is essentially used for further improving the quality of defogging images, but the guide filtering is not specially designed for solving the color distortion problem although the distortion problem is improved to a certain extent, so the color distortion problem in a sky area of a foggy image cannot be effectively solved, the resource consumption is large, and the instantaneity is limited. In addition, most of the prior art adopts a method for dividing a sky area and compensating transmittance to solve the problem of color distortion, but the existing method for dividing the sky area needs to calculate based on gray values and traditional Sobel operators, has low dividing precision, needs to additionally calculate gray values and the like, needs to traverse a foggy image for many times, and is difficult to balance dividing quality and hardware complexity, and the existing transmittance compensation method needs to additionally calculate a transmittance weight graph and the like besides the transmittance compensation, so that the implementation method is complex, the FPGA deployment is not easy, and the compensation effect is limited. Secondly, a dark channel prior defogging algorithm based on a physical model is realized by using an FPGA, and when an atmospheric light value is obtained, the dark channel prior defogging algorithm is easily interfered by a high-brightness area such as a white building in an image, so that a deviation exists in an obtaining result. Specifically, the atmospheric light value needs to be taken from the area with the highest fog concentration in the fog image, and the dark channel map of the fog image can reflect the fog concentration, so that the dark channel value of the pixel point is higher as the fog concentration is higher in general, and therefore the area with the highest fog concentration is selected according to the dark channel value of the pixel point. However, if an object like a white building exists in the foggy image, the dark channel value of the pixels in the white building area in the dark channel map is also high, so that the algorithm mists the area as an area with very dense foggy, and the obtained atmospheric light value is deviated. For example, when the dark channel value of the white building area is larger than