CN-115578722-B - License plate detection method based on inter-license plate collaborative learning mechanism

CN115578722BCN 115578722 BCN115578722 BCN 115578722BCN-115578722-B

Abstract

The invention belongs to the technical field of license plate detection, and relates to a license plate detection method based on a collaborative learning mechanism among license plates, which is based on strong consistency among license plate detection pictures in a license plate detection data set, the consistency among the network layer output characteristics of different license plate detection pictures is learned through a cooperative attention mechanism among scales, so that the interactivity among license plate characteristics of a network in a characteristic extraction stage is ensured, and sharing among common characteristics is realized through the interaction among the license plate characteristics. And then learning the consistency among the high-level output semantics of different license plate detection pictures through a semantic collaborative attention mechanism, thereby ensuring that a network can narrow the searching range of license plate features by virtue of semantic features, and finally fusing the relation among network semantic contexts through a semantic scale collaborative learning mechanism, so as to improve the license plate detection precision of complex scenes, and the method can be used for not only carrying out license plate detection under complex scenes, but also object segmentation under complex scenes and object detection under complex scenes.

Inventors

LIU HANSONG
SUN XIAOWEI
WANG YONG
WANG GUOQIANG
LIU RUI

Assignees

松立控股集团股份有限公司

Dates

Publication Date: 20260505
Application Date: 20221017

Claims (7)

1. A license plate detection method based on a collaborative learning mechanism between license plates is characterized by comprising the following steps: (1) Collecting license plate pictures under different complex scenes to construct a license plate detection data set, wherein each picture comprises license plate vertex coordinates and license plate character labeling information, and dividing the constructed license plate detection data set into a training set, a verification set and a test set; (2) Respectively inputting two license plate pictures in a training set into a basic network, outputting middle layer characteristics of the basic network as multi-scale characteristics, then converting side outputs of different layers of the network into characteristic layers with uniform dimensions through a convolution layer, and aggregating the multi-scale characteristics through downsampling and upsampling operations to obtain multi-scale aggregated characteristics; (3) Based on the multi-scale aggregation characteristics obtained in the step (2), the rich scale information interaction among license plates is learned through a scale-to-scale cooperative attention mechanism, the scale information interaction adopts a mode of modeling in a cooperative relation, namely, the scale information interaction is performed through learning a similarity matrix among the license plates, so that the consistency among the license plates is improved, and meanwhile, the background interference information of license plate areas is filtered out, so that a license plate refinement prospect area is obtained, and the specific process is as follows: Firstly, calculating a relation matrix among scales to obtain , Wherein, the Is a matrix multiplication, norm is a dimension-based normalization function, Is an L2 normalization function, normalizes the relation matrix values, Thinning the relation matrix to prevent noise data interference of information among license plates, and then performing multi-scale aggregation on the information 、 Enhancement is obtained 、 : , Wherein, the Based on multiplication between matrix elements The calculation mode of (a) is defined as follows, , Wherein the definition of the function The definition mode of the middle function is the same; (4) Solving the consistency of high-level areas among license plates by adopting a semantic collaborative attention mechanism, highlighting license plate areas with the same semantic meaning, and obtaining the most discriminative area characteristics of the license plates; (5) Adopting a semantic scale collaborative learning mechanism to perform collaborative learning on the license plate refined foreground region obtained in the step (3) and the license plate most discriminative region feature obtained in the step (4) to obtain refined license plate detection region features; (6) The refined license plate detection region features are converted into license plate classification information and license plate coordinate information, the aggregated features are combined with the initial features in a weighting mode, so that the common license plate region features are enhanced, and background interference information is filtered; (7) Training a license plate detection network by using the training set constructed in the step (1); (8) Testing the license plate detection network trained in the step (7) by using the test set constructed in the step (1), outputting the license plate category confidence and outputting the coordinate position information; (9) And recognizing characters in the license plate by means of an LSTM-based license plate recognition algorithm according to the output result of the license plate detection network, and outputting license plate character information to finish license plate detection.
2. The license plate detection method based on the inter-license plate collaborative learning mechanism according to claim 1, wherein the basic network in step (2) is VggNet networks, and the convolution kernel of the convolution layer is The multi-scale characteristics of the two pictures are respectively And According to multi-scale features The obtained multi-scale polymerization characteristics: Wherein Con represents a feature superposition operation and/represents an up-sampling or down-sampling operation on different scale features, for The same operation is adopted to obtain multi-scale polymerization characteristics 。
3. The license plate detection method based on the inter-license plate collaborative learning mechanism according to claim 2, wherein the specific process of step (4) is as follows: Firstly, calculating a relation matrix among semantics to obtain: , Further to semantic features And Enhancement results in: , The definition of (c) is as follows, , Wherein each symbol is defined The definition of the functions is the same.
4. The license plate detection method based on the inter-license plate collaborative learning mechanism according to claim 3, wherein the specific process of step (5) is as follows: , , Calculation mode and of (2) The same way of calculation.
5. The license plate detection method based on the inter-license plate collaborative learning mechanism according to claim 4, wherein the process of weighting and fusing the aggregated features and the initial features in the step (6) is as follows: , The calculation mode of (2) is as follows: 。
6. the license plate detection method based on the inter-license plate collaborative learning mechanism according to claim 5, wherein the specific process of training the license plate detection network in step (7) is to train two pieces of picture data of a training set And Inputting the license plates into a network to obtain license plates respectively Category confidence of (v) And returning the coordinate position And license plate Category confidence of (v) And returning the coordinate position Category confidence And Judging whether the current branch prediction is license plate or not, and returning to the coordinate position And For four vertex coordinates of the license plate, and adopting FocalLoss to calculate the license plate And Loss, smooth L1 Loss calculation license plate And After 55 complete training sets and training iterations, the model parameters with highest preservation precision are the trained model parameters, wherein W, H and N are the width, the height and the number of pictures respectively.
7. The license plate detection method based on the inter-license plate collaborative learning mechanism according to claim 6, wherein the specific process of testing the license plate detection network in the step (8) is to test the picture data of the test set And Inputting the model parameters into a network, loading the model parameters trained in the step (7), and outputting the model parameters from the network to obtain the license plate category confidence coefficient And returning the coordinate position And filtering out the license plate with low confidence through a threshold value, and finally deleting the redundant license plate detection frame output by the network by using non-maximum inhibition.

Description

License plate detection method based on inter-license plate collaborative learning mechanism Technical Field The invention belongs to the technical field of license plate detection, and relates to a license plate detection method based on a collaborative learning mechanism between license plates. Background Particularly in the field of license plate recognition, more and more algorithms based on artificial intelligence are proposed, and very excellent performance is achieved, so that the method is widely applied to life scenes. The traditional license plate recognition algorithm is mostly a manually designed feature extraction algorithm, is very limited in application in complex and changeable actual life scenes, particularly in very difficult scenes (rainy and snowy weather, high/low contrast), so that the performance of the license plate detection algorithm is very limited, as a large-scale data set is recognized by the license plate, the traditional method is gradually replaced by a deep learning-based method, but the deep learning-based method still has a plurality of problems, because the deep learning-based method belongs to a data driving mode, the performance of the deep learning-based method depends on the diversity of the data set, the conventional algorithm is insufficient for mining the data set, only single picture detection is considered, the key point is mainly the design of a network layer to improve the detection precision, and the single picture feature mining easily causes the diversity information loss of the data set and the insufficient mining of mutual information of the license plate in the data set. Therefore, aiming at the license plate detection technology in the complex scene, a new license plate detection method is needed to improve the precision of license plate detection in the complex scene. Disclosure of Invention The invention aims to overcome the defects of the prior art and designs and provides a license plate detection method based on a collaborative learning mechanism between license plates. In order to achieve the above purpose, the present invention firstly extracts output features of different layers as multi-scale features through a backbone network, learns consistency between network layer output features of different license plate detection pictures through inter-scale cooperative attention mechanisms on the basis of the multi-scale features, thereby guaranteeing interactivity between license plate features of the network in a feature extraction stage, learns consistency between high layer output semantics of different license plate detection pictures through semantic cooperative attention mechanisms, thereby guaranteeing that the network can reduce a searching range of license plate features by means of semantic features, and finally, fuses relations between network semantic contexts through semantic scale cooperative learning mechanisms, and improves license plate detection precision of complex scenes, and the present invention specifically comprises the following steps: (1) Collecting license plate pictures under different complex scenes to construct a license plate detection data set, wherein each picture comprises license plate vertex coordinates and license plate character labeling information, and dividing the constructed license plate detection data set into a training set, a verification set and a test set; (2) Respectively inputting two license plate pictures in a training set into a basic network, outputting middle layer characteristics of the basic network as multi-scale characteristics, then converting side outputs of different layers of the network into characteristic layers with uniform dimensions through a convolution layer, and aggregating the multi-scale characteristics through downsampling and upsampling operations to obtain multi-scale aggregated characteristics; (3) Based on the multi-scale aggregation characteristics obtained in the step (2), the rich scale information interaction among license plates is learned through a scale-to-scale cooperative attention mechanism, and the scale information interaction adopts a mode of modeling a cooperative relationship, namely, the scale information interaction is performed through a similarity matrix among the learning license plates, so that the consistency among the license plates is improved, and meanwhile, the background interference information of license plate areas is filtered, so that a license plate refinement prospect area is obtained; (4) Solving the consistency of high-level areas among license plates by adopting a semantic collaborative attention mechanism, highlighting license plate areas with the same semantic meaning, and obtaining the most discriminative area characteristics of the license plates; (5) Adopting a semantic scale collaborative learning mechanism to perform collaborative learning on the license plate refined foreground region obtained in the step (3) and the license plate