CN-122023155-A - License plate image quality enhancement method and system based on black light camera

CN122023155ACN 122023155 ACN122023155 ACN 122023155ACN-122023155-A

Abstract

The invention discloses a license plate image quality enhancement method and system based on a black light camera, which comprises the steps of obtaining a license plate image under natural light, embedding GlareDecomNet a convolution model of a point spread function which is built in advance based on lens parameters and light source intensity of the black light camera to generate an image preprocessing model, inputting the license plate image into the image preprocessing model to remove glare of the license plate image, combining a task perception ISP (Internet of things) controller which is built in advance with OCR (optical character Unit), realizing self-adaptive processing of a license plate area and a background area of the license plate image, carrying out weighted fusion on the license plate images of a plurality of continuous frames according to credibility weights based on time sequence context enhancement and motion compensation, and realizing self-adaptive noise reduction of a stationary area and a motion area of the license plate image, thereby completing quality enhancement of the license plate image. The problems of light pollution, low operation and maintenance efficiency and high energy consumption in the prior imaging technology are solved.

Inventors

HU TIANYU
ZHANG FAKUAN
ZHANG YANDUO
WANG GANG
ZHAO YONGSHENG
GUO XIAOBIN
ZHANG ZHAO
WU XIAOFENG
LIU GANG
XU JIANMING

Assignees

北京云星宇交通科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260116

Claims (10)

1. A license plate image quality enhancement method based on a black light camera is characterized by comprising the following steps: Acquiring license plate images under natural light; Embedding GlareDecomNet a convolution model of a point spread function which is constructed in advance based on lens parameters and light source intensity of a black light camera into the model to generate an image preprocessing model; combining a task perception ISP controller constructed in advance with OCR to realize self-adaptive processing of license plate areas and background areas of the license plate images; And based on time sequence context enhancement and motion compensation, weighting and fusing the license plate images of a plurality of continuous frames according to the credibility weight to realize self-adaptive noise reduction of the license plate image static region and the motion region, thereby completing the quality enhancement of the license plate image.
2. The method according to claim 1, wherein the convolution model of the spatially-varying point spread function is specifically a convolution model PSF of the spatially-varying point spread function, which is pre-constructed based on lens parameters of the black-light camera and light source intensity: Wherein the method comprises the steps of For the light source center, σ is estimated from the luminance peak, and (x, y) is the pixel coordinates on the image plane.
3. The method of claim 1, wherein embedding GlareDecomNet a convolution model of a pre-constructed spatially varying point spread function based on lens parameters and light source intensity of a black-light camera into the model to generate an image pre-processing model, inputting the license plate image into the image pre-processing model to remove glare from the license plate image, comprises: taking a convolution model of the point spread function with spatial variation as a differential regular term, embedding GlareDecomNet the convolution model into the model, and generating an image preprocessing model; inputting the license plate image into the image preprocessing model, removing glare of the license plate image, and outputting a license plate image only retaining the reflective property of the license plate; The loss function of the image preprocessing model is as follows: Wherein Iobs is an original image, PSF is an priori map, G is a glare layer, S is a scene layer, lambda 1 is a physical constraint weight, and lambda 2 is a scene sparse weight.
4. The method of claim 1, wherein the combining the pre-built task-aware ISP controller with OCR to achieve adaptive processing of license plate regions and background regions of the license plate image comprises: constructing a network model combining task-aware ISP controller and OCR based on TA-ISP Net; in the network model training stage, the task-aware ISP controller returns ISP parameter vectors Inputting a differentiable ISP simulator; The original image and the ROI thermodynamic diagram of the preprocessed license plate image are sent into an OCR network, the OCR network calculates the cross entropy loss of characters, the OCR accuracy is used as an optimization target, and the TA-ISP Net and the OCR network are updated to complete training of the model; after training, freezing OCR, and only deploying TA-ISP Net; And inputting the preprocessed license plate image into a TA-ISPNet, and outputting the license plate image with high contrast in a license plate region and low noise in a background region in a region self-adaptive processing mode.
5. The method of claim 1, wherein weighting and fusing the license plate images of a plurality of consecutive frames according to confidence weights based on temporal context enhancement and motion compensation to achieve adaptive noise reduction for the license plate image still region and motion region comprises: Calculating an optical flow field of 3 frames immediately preceding a current frame in the license plate image of a plurality of consecutive frames using a PWC-Net reduced version model ; Mapping the optical flow to the current frame coordinates for each frame of candidate license plate region; setting credibility weight: Where β is a sensitivity coefficient, it is a current frame image, it-i is a current i-th frame image; according to the following formula, weighting and fusing the license plate images of a plurality of continuous frames according to the credibility weight to realize the self-adaptive noise reduction of the static area and the moving area of the license plate images, Where wi is the confidence level and Σwi is the normalization factor.
6. A license plate image quality enhancement system based on a black-light camera, comprising: The image acquisition module is used for acquiring license plate images under natural light; The preprocessing module is used for embedding GlareDecomNet a convolution model of a point spread function based on the lens parameters and the light source intensity of the black light camera and constructed in advance to generate an image preprocessing model; the region self-adaptive processing module is used for combining a task perception ISP controller constructed in advance with OCR to realize self-adaptive processing of a license plate region and a background region of the license plate image; And the self-adaptive noise reduction module is used for carrying out weighted fusion on the license plate images of a plurality of continuous frames according to the credibility weight based on time sequence context enhancement and motion compensation, so as to realize the self-adaptive noise reduction of the static area and the motion area of the license plate images, thereby completing the quality enhancement of the license plate images.
7. The system of claim 6, wherein the convolution model of the spatially varying point spread function is specifically a convolution model PSF of the spatially varying point spread function that is pre-constructed based on lens parameters of the black-light camera and light source intensity: Wherein the method comprises the steps of For the light source center, σ is estimated from the luminance peak, and (x, y) is the pixel coordinates on the image plane.
8. The system of claim 6, wherein the preprocessing module comprises: The preprocessing model generation submodule is used for taking a convolution model of a spatially-changed point spread function as a differential regularization term, embedding GlareDecomNet the model and generating an image preprocessing model; The glare removing sub-module is used for inputting the license plate image into the image preprocessing model, removing the glare of the license plate image and outputting the license plate image only retaining the reflective property of the license plate; The loss function of the image preprocessing model is as follows: Wherein Iobs is an original image, PSF is an priori map, G is a glare layer, S is a scene layer, lambda 1 is a physical constraint weight, and lambda 2 is a scene sparse weight.
9. The system of claim 6, wherein the region adaptive processing module comprises: the network model construction submodule is used for constructing a network model combining task perception ISP controller and OCR based on TA-ISP Net; the vector input sub-module is used for the training stage of the network model, and the task-aware ISP controller returns ISP parameter vectors Inputting a differentiable ISP simulator; The training sub-module is used for sending the original image and the preprocessed ROI thermodynamic diagram of the license plate image into an OCR network, and the OCR network calculates the cross entropy loss of characters, takes OCR accuracy as an optimization target, and updates the TA-ISP Net and the OCR network to complete training of the model; the deployment sub-module is used for freezing OCR after training is finished and only deploying TA-ISP Net; The region self-adaptive processing sub-module is used for inputting the preprocessed license plate image into the TA-ISPNet, outputting the license plate image with high contrast in the license plate region and low noise in the background region.
10. The system of claim 6, wherein the adaptive noise reduction module comprises: an optical flow field calculation sub-module for calculating an optical flow field of 3 frames immediately before a current frame in the license plate image of a plurality of continuous frames using a PWC-Net reduced version model ; The optical flow mapping sub-module is used for mapping the optical flow to the current frame coordinates for each frame of candidate license plate region; setting credibility weight: Where β is a sensitivity coefficient, it is a current frame image, it-i is a current i-th frame image; the weighting fusion sub-module is used for carrying out weighting fusion on the license plate images of a plurality of continuous frames according to the following formula and credibility weight to realize the self-adaptive noise reduction of the static area and the moving area of the license plate images, Where wi is the confidence level and Σwi is the normalization factor.

Description

License plate image quality enhancement method and system based on black light camera Technical Field The invention relates to the technical field of light processing, in particular to a license plate image quality enhancement method and system based on a black light camera. Background At present, the ETC portal adopts a video license plate recognition mode of license plate recognition and light supplementing lamps (one main one standby) as a main auxiliary method for judging the running of the vehicle. Meanwhile, due to the fact that a large number of light supplementing lamps are added, great danger is caused to the driving process of a driver, and strong social influence is brought. Meanwhile, the management department considers reducing complaints to select to turn off the light supplementing lamp, so that the value of license plate recognition cannot be effectively exerted. According to statistics, only license plate recognition equipment used on the expressway is not less than 50 ten thousand sets, and scenes such as a toll gate, a pre-transaction portal, an ETC portal, a tunnel, a service area and the like are covered. The national police traffic police and the parking lot are more not lower than ten million sets. The light pollution and the energy consumption of the LED light filling lamp are not counted each year. Because the light supplementing lamp is completely relied on at night, the light supplementing lamp can stimulate eyes of people when the license plate is lightened, and serious blindness is easy to cause, so that a large number of traffic hidden troubles and complaints are caused. Because the different time of shining on the day in different seasons is different, the light filling lamp must constantly follow the time change over switch in the day, and the day can light be too dark in the overcast and rainy day of sleet day simultaneously, also needs interim debugging, causes great maintenance difficulty, promotes maintenance cost. The light supplementing lamp generally has power consumption of about 50 watts, the existing camera generally has power consumption of 10 watts, and the long-term operation can bring larger power supply cost. Therefore, the development of a new generation of video camera which does not need a light supplementing lamp, does not need a coil car detector and supports domestic credit and trauma and AI algorithm expansion by a core component becomes a main target of current work. Disclosure of Invention Aiming at the technical problems, the invention provides a license plate image quality enhancement method based on a black light camera, which comprises the following steps: Acquiring license plate images under natural light; Embedding GlareDecomNet a convolution model of a point spread function which is constructed in advance based on lens parameters and light source intensity of a black light camera into the model to generate an image preprocessing model; combining a task perception ISP controller constructed in advance with OCR to realize self-adaptive processing of license plate areas and background areas of the license plate images; And based on time sequence context enhancement and motion compensation, weighting and fusing the license plate images of a plurality of continuous frames according to the credibility weight to realize self-adaptive noise reduction of the license plate image static region and the motion region, thereby completing the quality enhancement of the license plate image. Further, the convolution model of the spatially-varying point spread function is specifically a convolution model PSF of the spatially-varying point spread function, which is constructed in advance based on lens parameters of the black-light camera and light source intensity: Wherein the method comprises the steps of For the light source center, σ is estimated from the luminance peak, and (x, y) is the pixel coordinates on the image plane. Further, embedding GlareDecomNet a convolution model of a point spread function based on the lens parameters and the light source intensity of the black light camera and constructed in advance to generate an image preprocessing model, inputting the license plate image into the image preprocessing model, and removing the glare of the license plate image, wherein the method comprises the following steps: taking a convolution model of the point spread function with spatial variation as a differential regular term, embedding GlareDecomNet the convolution model into the model, and generating an image preprocessing model; inputting the license plate image into the image preprocessing model, removing glare of the license plate image, and outputting a license plate image only retaining the reflective property of the license plate; The loss function of the image preprocessing model is as follows: Wherein Iobs is an original image, PSF is an priori map, G is a glare layer, S is a scene layer, lambda 1 is a physical constraint weight, and lambda 2 is