Search

CN-121982883-A - Illegal parking detection method based on unmanned aerial vehicle visual recognition

CN121982883ACN 121982883 ACN121982883 ACN 121982883ACN-121982883-A

Abstract

The invention discloses a method for detecting illegal parking based on visual identification of an unmanned aerial vehicle, which belongs to the technical field of illegal parking detection, wherein the unmanned aerial vehicle cruises according to preset parameters, a stationary vehicle in an illegal area is detected in real time by using an onboard vehicle detection model, when a target vehicle is detected, the unmanned aerial vehicle adjusts the gesture, photographs license plates of the vehicle from two different angles to obtain a plurality of groups of license plate images, performs feature matching and image alignment on the plurality of groups of license plate images, fuses the plurality of groups of license plate images to generate a complete license plate image, and extracts license plate feature information for illegal judgment based on the fused complete license plate image. The method and the device can effectively improve the accuracy and the instantaneity of vehicle detection, the integrity of license plate images and the accuracy of subsequent character recognition.

Inventors

  • SUN HUI
  • Guo Zhina
  • Lu Zhengbo
  • XU AICUI
  • HAN ZHIFENG

Assignees

  • 山东科技大学

Dates

Publication Date
20260505
Application Date
20260109

Claims (9)

  1. 1. The method for detecting the illegal parking based on the unmanned aerial vehicle visual recognition is characterized by comprising the following steps of: Step 1, an unmanned aerial vehicle cruises according to preset parameters, and stationary vehicles in a violation area are detected in real time by using an airborne vehicle detection model; Step 2, when a target vehicle is detected, the unmanned aerial vehicle adjusts the gesture, and the license plates of the vehicle are shot from two different angles to obtain a plurality of groups of license plate images; step 3, performing feature matching and image alignment on the plurality of groups of license plate images, and fusing to generate a complete license plate image; And 4, extracting license plate characteristic information for violation judgment based on the integrated complete license plate image.
  2. 2. The method for detecting the illegal parking based on the unmanned aerial vehicle visual recognition according to claim 1, wherein in the step 1, a vehicle detection model is combined with an optical flow method to judge whether a vehicle is stationary, the vehicle detection model is YOLOv-PCW model, a backbone network of YOLOv is replaced by a PP-LCNet network, the network is formed by stacking five PP-LCNet Block layers, and each PP-LCNet Block adopts a depth separable convolution module and is responsible for extracting image features of different layers; The neck of YOLOv is embedded with an F-CBAM attention mechanism module, the F-CBAM attention mechanism module is matched with A2C2F and C3k2 modules of the neck to carry out channel and space double weighting on the fused multi-scale characteristics, the F-CBAM attention mechanism module comprises a channel attention sub-module and a space attention sub-module, the channel attention sub-module is used for screening channel information which is critical to violation detection in the neck fusion characteristics, and the space attention sub-module is used for focusing a space region where a target in a neck characteristic diagram is located; In the channel attention submodule, firstly, respectively carrying out self-adaptive average pooling and self-adaptive maximum pooling on an input feature map x to obtain two 1 multiplied by 1 feature maps And Will (i) be And Channel compression is carried out through a first 1x1 convolution layer respectively to obtain And Then nonlinear transformation is carried out through SiLU activation functions to obtain And Finally, respectively carrying out channel recovery through a second 1x1 convolution layer to obtain And Will (i) be And Element addition is carried out, and a final channel attention weight matrix is generated through a Sigmoid function Channel attention weight matrix to be generated Multiplying the original input feature map x element by element to obtain a feature map with enhanced channel attention The shape is , In order to be of a height, the height, In the form of a width, the width, The number of channels; In the space attention sub-module, the output characteristic diagram of the channel attention module Extracting spatial features with different geometric meanings from four independent paths respectively, wherein the first path is to extract features in horizontal direction, and perform adaptive average pooling in horizontal direction to obtain Then to Dimension expansion and mean processing are carried out to obtain And path two is to extract the vertical direction characteristics, and perform self-adaptive average pooling in the vertical direction to obtain Then to Dimension expansion and mean processing are carried out to obtain And extracting the maximum value characteristic of the channel in the third path, and taking the maximum value of the channel dimension to obtain Path four is used for extracting the average value characteristics of the channel, and the average value of the channel dimension is obtained Splicing the features extracted by the four paths in the channel dimension to obtain Sequentially performing depth convolution and point-by-point convolution to obtain ; Will be Carrying out batch normalization processing, generating a final space attention weight matrix through a Sigmoid function, and enhancing the generated space attention weight matrix and the channel attention characteristic diagram Multiplying element by element to obtain final output 。
  3. 3. The unmanned aerial vehicle vision recognition-based illegal parking detection method according to claim 1, wherein the vehicle detection model is trained by a Wk-IoU loss function, a Wk-IoU loss function The expression is: ; Wherein, the As a basic cross ratio, the expression is: ; Wherein, the For the intersection area of the predicted frame and the real frame in the target detection, i.e. the area of the overlapping part of the two frames, In order to predict the height of the frame, In order to predict the width of the frame, For the width of the real frame, For the height of the real frame, Is a numerical stability term; as a dynamic scale factor, the expression is: ; Wherein, the Is the parameter of the ultrasonic wave to be used as the ultrasonic wave, Is that Moving average, expressed as: ; Wherein, the As the momentum coefficient of the magnetic field, For iteration at step t The moving average value of the average value is calculated, Is an average calculation; For comprehensive weight, the expression is: ; Wherein, the For a small target weight factor: ; for occlusion scene weight factors: ; Is an aspect ratio weight factor: 。
  4. 4. the method for detecting the illegal parking based on the visual recognition of the unmanned aerial vehicle according to claim 1, wherein the step 3 comprises the following substeps: Step 3.1, preprocessing a plurality of groups of license plate images, wherein the preprocessing comprises denoising, graying and binarization; Step 3.2, based on the preprocessed images, respectively adopting DeepLab V3+ semantic segmentation models to segment license plate region images in the images; step 3.3, matching a plurality of image feature points through a SuperGlue model aiming at the plurality of segmented license plate region images, and establishing inter-image association; step 3.4, calculating a homography matrix by adopting a RANSAC algorithm according to the matched characteristic points, and aligning each license plate region image to the same coordinate space; And 3.5, carrying out Alpha weighted fusion on the aligned license plate region images to generate a complete license plate image.
  5. 5. The method for detecting the illegal parking based on the unmanned aerial vehicle visual recognition, which is disclosed by claim 4, is characterized in that in the step 3.2, the preprocessed vehicle images shot at different angles are input into a DeepLab V3+ semantic segmentation model to obtain license plate images at different angles, wherein the DeepLab V3+ semantic segmentation model comprises an encoder module and a decoder module, the encoder module captures multi-scale context information through cavity convolution, and the decoder is used for recovering space details so as to realize pixel level segmentation of a license plate region; The expression of the cavity convolution is as follows: ; Wherein, the To output the characteristic diagram at the first The value of the individual position(s), In order for the expansion rate to be high, For the sampling location of the input feature, Is convolution kernel at the first Weights for individual locations.
  6. 6. The method for detecting illegal parking based on unmanned aerial vehicle visual recognition according to claim 4, wherein in the step 3.3, the SuperGlue model firstly constructs feature points of two license plate region images into two graph structures respectively, and each feature point As a graph node Then carrying out iterative optimization on the matching relation of the feature points through a message transfer mechanism of multiple rounds of iteration, wherein a message transfer formula is as follows: ; Wherein, the Represent the first Node during round iteration Is used for the feature vector of (a), Representing nodes Sum node The edge attributes of the two-dimensional space, An update function representing the edge feature, Is a multi-layer sensing machine, which is a multi-layer sensing machine, Is a node Is a neighbor node set; the updated node characteristics are obtained by calculating the similarity scores among the nodes Obtaining a similarity score matrix , The expression is: ; Wherein, the Representing the operation of the vector dot product, Is the total number of rounds of message delivery; Finally, matching optimization is carried out, and a Sinkhorn algorithm is used for carrying out similarity score matrix Conversion to a bi-directionally normalized probability matrix Representing the final matching probability, probability matrix Elements of (a) Representing feature points And Matching probabilities of (a); according to the matching probability matrix If and only if At the same time be the first Maximum sum of rows Only when the maximum value of the column is determined And forming a matching point pair set for the effective matching point pair.
  7. 7. The method for detecting the illegal parking based on the unmanned aerial vehicle visual recognition according to claim 4, wherein in the step 3.4, the method comprises the following steps: Step 3.4.1, randomly extracting 4 non-collinear matching point pairs from the matching point sets of the two images, and calculating a candidate homography matrix : ; Wherein, the For the matching point of the first map, For the corresponding point of the second graph, expanding the formula into a linear equation set, substituting the linear equation set into four matching point pairs, and solving to obtain a3 multiplied by 3 candidate homography matrix ; Step 3.4.2. All matching points in the first image are processed Using Mapping the two images to calculate the reprojection error of each matching point pair: ; Wherein the feature points And Respectively, the matching points and the corresponding points in the two images, when If the number of the matching points is smaller than the threshold t, the matching point pair is identified as an inner point, otherwise, the matching point pair is identified as an outer point; step 3.4.3, judging whether the proportion of the interior points exceeds a preset value d, if not, repeating the steps 3.4.1 and 3.4.2, if so, terminating iteration, and outputting the current optimal homography matrix ; Step 3.4.4 according to The first image is transformed to a coordinate system of the second image.
  8. 8. The method for detecting the illegal parking based on the unmanned aerial vehicle visual recognition according to claim 4, wherein in the step 3.5, the pixel values of the two images are weighted and summed through the weight coefficient to obtain a fused license plate region image, and the expression is as follows: ; Wherein, the Finger fused image at coordinates The pixel value at which it is located, And Respectively refers to the coordinates of two input images The pixel value at which it is located, Is a weighting coefficient; and correcting the fused license plate region image to a standard rectangular plane through perspective transformation.
  9. 9. The method for detecting the illegal parking based on the unmanned aerial vehicle visual recognition according to claim 4, wherein in the step 4, preprocessing operation is performed on the corrected license plate region image, including graying, binarization and denoising; dividing characters in the preprocessed image one by one, obtaining information in a minimum rectangular frame of the image by using a cv2.Bound-ingRect () function, ordering license plate number information, and finally displaying a license plate character division map; Aiming at the characteristics of license plate characters, namely, the first position is Chinese province abbreviation, the second position is English letter, the other positions are English and digital combination, a template dividing matching strategy is adopted, namely, the first position calls a Chinese character template library for comparison, the second position uses an English character template, the other positions adopt an English and digital mixed template library, in the specific matching process, firstly, a template path and a picture to be identified are read, the template is subjected to uniform format and image binarization, then the size of the image to be detected is extracted, the template is scaled to the same size, and finally, the identification of license plate numbers is completed through a template matching algorithm.

Description

Illegal parking detection method based on unmanned aerial vehicle visual recognition Technical Field The invention belongs to the technical field of illegal parking detection, and particularly relates to an unmanned aerial vehicle visual recognition-based illegal parking detection method. Background Automobiles are increasingly owned in cities as vehicles. There are fewer and fewer free locations in the city for parking the car, thereby creating many incidents of illicit parking. At present, the detection mode of illegal parking is mainly detected by a manual mode. The detection mode requires a user to manually monitor all places incapable of stopping for a long time, and also requires staff to paste a ticket on a vehicle, so that the user cannot continuously patrol all places and positions capable of prohibiting stopping for 24 hours under the influence of weather. Meanwhile, parking in the illegal parking area is messy, the existing visual recognition algorithm has the problems of missing detection, false detection and the like in traffic scene application, and particularly, the detection precision of small target vehicles and shielding vehicles in complex scenes is insufficient, and the detection requirements of high precision and real-time performance are difficult to meet, so that the traditional mode and the existing algorithm have great limitations, and the requirements of staff are difficult to meet. The main stream visual recognition algorithm is YOLOv, when the vehicles are dense, the license plate of the target vehicle is easily blocked by front and rear vehicles and roadside obstacles, or license plate characters are incomplete due to overlarge shooting angle (such as side shooting) of an unmanned aerial vehicle, and the follow-up recognition requirement cannot be met. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a method for detecting illegal parking based on unmanned aerial vehicle visual recognition, which has reasonable design, solves the defects in the prior art and has good effect. In order to achieve the above purpose, the following technical scheme is adopted: a method for detecting illegal parking based on unmanned aerial vehicle visual recognition comprises the following steps: Step 1, an unmanned aerial vehicle cruises according to preset parameters, and stationary vehicles in a violation area are detected in real time by using an airborne vehicle detection model; Step 2, when a target vehicle is detected, the unmanned aerial vehicle adjusts the gesture, and the license plates of the vehicle are shot from two different angles to obtain a plurality of groups of license plate images; step 3, performing feature matching and image alignment on the plurality of groups of license plate images, and fusing to generate a complete license plate image; And 4, extracting license plate characteristic information for violation judgment based on the integrated complete license plate image. Further, in the step 1, the vehicle detection model is combined with the optical flow method to determine whether the vehicle is stationary, the vehicle detection model is YOLOv-PCW model, the backbone network of YOLOv is replaced by a PP-LCNet network, the network is formed by stacking five PP-LCNet Block layers, and each PP-LCNet Block adopts a depth separable convolution module to extract image features of different layers; The neck of YOLOv is embedded with an F-CBAM attention mechanism module, the F-CBAM attention mechanism module is matched with A2C2F and C3k2 modules of the neck to carry out channel and space double weighting on the fused multi-scale characteristics, the F-CBAM attention mechanism module comprises a channel attention sub-module and a space attention sub-module, the channel attention sub-module is used for screening channel information which is critical to violation detection in the neck fusion characteristics, and the space attention sub-module is used for focusing a space region where a target in a neck characteristic diagram is located; In the channel attention submodule, firstly, respectively carrying out self-adaptive average pooling and self-adaptive maximum pooling on an input feature map x to obtain two 1 multiplied by 1 feature maps AndWill (i) beAndChannel compression is carried out through a first 1x1 convolution layer respectively to obtainAndThen nonlinear transformation is carried out through SiLU activation functions to obtainAndFinally, respectively carrying out channel recovery through a second 1x1 convolution layer to obtainAndWill (i) beAndElement addition is carried out, and a final channel attention weight matrix is generated through a Sigmoid functionChannel attention weight matrix to be generatedMultiplying the original input feature map x element by element to obtain a feature map with enhanced channel attentionThe shape is,In order to be of a height, the height,In the form of a width, the width,The number of channels; In the