CN-122024116-A - Road vehicle speed estimation method and system based on unmanned aerial vehicle inspection

CN122024116ACN 122024116 ACN122024116 ACN 122024116ACN-122024116-A

Abstract

The application relates to a pavement vehicle speed estimation method and system based on unmanned aerial vehicle inspection, which are characterized in that real-time video data of a pavement is obtained through an onboard camera of an unmanned aerial vehicle, a trained target detection model is adopted to detect a vehicle target in the real-time video data, a vehicle detection frame and the confidence coefficient thereof are generated, a multi-target tracking algorithm is adopted to track the vehicle detection frame and the confidence coefficient thereof in frame images of the real-time video data, a target track of the vehicle target is obtained, a dense optical flow field between two adjacent frame images is constructed based on a dense optical flow algorithm, abnormal points are screened according to the motion state of the vehicle detection frame and the background in the real-time video data, an effective background pair is obtained after abnormal points are removed, a homography matrix is obtained according to the effective background pair, motion compensation is carried out on the target track according to the homography matrix, a vehicle real track is generated, and the speed of the vehicle target is estimated according to the vehicle real track.

Inventors

HUANG ZHENG
LIU TIANLONG
Shi Qiaomu

Assignees

杭州靖安科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260415

Claims (10)

1. The road vehicle speed estimation method based on unmanned aerial vehicle inspection is characterized by comprising the following steps of: Detecting a vehicle target in the real-time video data by adopting a trained target detection model to generate a vehicle detection frame and the confidence coefficient thereof; Tracking a vehicle detection frame and the confidence coefficient thereof in a frame image of the real-time video data through a multi-target tracking algorithm to obtain a target track of the vehicle target; constructing a dense light flow field between two adjacent frames of images based on a dense light flow algorithm, screening abnormal points of the dense light flow field according to the motion state of the background in the vehicle detection frame and the real-time video data, removing the abnormal points to obtain effective background pairs, calculating according to the effective background pairs to obtain a homography matrix, performing motion compensation on the target track according to the homography matrix, and generating a real track of the vehicle; an actual diagonal length and a pixel diagonal length of the vehicle target are obtained, and a speed of the vehicle target is estimated based on the vehicle actual trajectory, the actual diagonal length, and the pixel diagonal length.
2. The method of claim 1, wherein the constructing a dense optical flow field between two adjacent frames of images based on a dense optical flow algorithm, and performing outlier screening on the dense optical flow field according to the motion states of the background in the vehicle detection frame and the real-time video data, and removing outliers to obtain an effective background pair, comprises: Calculating the motion light flow of each pixel point between two continuous frames of images by adopting Gunnar Farneback algorithm, constructing the dense light flow field by the motion light flow of all the pixel points, and extracting pixel matching point pairs from the dense light flow field; Screening the pixel matching point pairs once by taking the vehicle detection frame as a mask, and removing the pixel matching point pairs in the vehicle detection frame to obtain primary selected pixel matching point pairs; And analyzing the background motion state of the real-time video data according to the initially selected pixel matching point pairs, and carrying out secondary screening on the initially selected pixel matching point pairs through a random sampling consistency algorithm under the condition that the background motion state is judged to be motion, and eliminating abnormal pixel matching point pairs in the background to obtain effective background point pairs.
3. The method of claim 2, wherein said analyzing the background motion state of the real-time video data from the initially selected pair of pixel matching points comprises: acquiring the optical flow amplitude and the optical flow direction of each pixel point according to the initially selected pixel matching point pair, and calculating the consistency of the average optical flow amplitude and the optical flow direction of the two continuous frames of images; acquiring a preset optical flow threshold value and a preset direction threshold value, if the average optical flow amplitude value is smaller than the preset optical flow threshold value and the optical flow direction consistency is smaller than the preset direction threshold value, judging that the background motion state is a stationary state, otherwise, judging that the background motion state is a motion state.
4. The method of claim 1, wherein the calculating the homography matrix according to the effective background pair, and the motion compensating the target track according to the homography matrix, generating a real track of the vehicle, includes: calculating scaling factors, twiddle factors, translation factors and transformation factors between two continuous frames of images according to the effective background pairs, and constructing the homography matrix according to the scaling factors, twiddle factors, translation factors and transformation factors; Respectively marking the continuous two-frame images as a current frame image and a previous frame image, acquiring a target track of the vehicle target in the previous frame image, mapping the target track into the current frame image according to the homography matrix, and acquiring a compensation track of the vehicle detection frame in the current frame image; And generating a real track of the vehicle target according to all the compensation tracks.
5. The method of claim 1, wherein the acquiring the actual diagonal length and the pixel diagonal length of the vehicle target and estimating the speed of the vehicle target from the vehicle actual trajectory, the actual diagonal length, and the pixel diagonal length comprises: acquiring pixel displacement of a vehicle target within a preset frame number from the real track of the vehicle, and acquiring the diagonal length of the pixels of the vehicle target in an image through a dynamically maintained Gaussian model; Acquiring the actual diagonal length of the vehicle target, and calculating a distance mapping parameter between a pixel space and the real world according to the actual diagonal length and the pixel diagonal length; calculating to obtain pixel displacement time according to the preset frame number and the video frame rate of the real-time video data; and calculating the vehicle instantaneous speed of the vehicle target according to the pixel displacement, the pixel displacement time and the distance mapping parameter.
6. The method of claim 5, wherein calculating a distance mapping parameter between pixel space and the real world from the actual diagonal length and the pixel diagonal length comprises: Constructing a Gaussian model, and initializing the Gaussian model by using the pixel diagonal length average value of all the vehicle detection frames in the first frame image; detecting the pixel diagonal length of a vehicle detection frame in the real-time video data frame by frame, and iteratively updating parameters of the Gaussian model in real time through the latest pixel diagonal length so as to obtain the statistical average value of the pixel length of the vehicle diagonal; And dynamically updating the distance mapping parameter according to the ratio of the actual diagonal length of the vehicle target to the statistical mean value.
7. The method of claim 1, wherein the training process of the object detection model comprises: acquiring historically acquired traffic image data, and labeling vehicle targets in the traffic image data to obtain a training data set; Constructing a target detection model to be trained by adopting a YOLO11 network structure comprising a backbone network, a neck structure and a head module, wherein the backbone network is sequentially provided with a convolution layer, a C3K2 module, an SPPF module and a C2PSA module, and is used for extracting multi-scale characteristics of an input frame image; Inputting the training data set into the target detection model to be trained, and adopting a gradient descent optimization algorithm to iteratively update trainable parameters in the backbone network, the neck structure and the head module until a preset loss function converges, so as to obtain a trained target detection model.
8. The method according to claim 1, wherein the tracking the vehicle detection frame and the confidence thereof in the frame image of the real-time video data by the multi-target tracking algorithm to obtain the target track of the vehicle target comprises: Acquiring vehicle detection frames and the confidence coefficient thereof output by the target detection model, and screening all the vehicle detection frames according to a preset confidence coefficient threshold value to obtain a low score frame set and a high score frame set; Predicting all real tracks in a current track set by using Kalman filtering, predicting potential positions of the real tracks in a next frame of image, and updating the real tracks according to the potential positions to obtain predicted tracks; Calculating IoU values of each vehicle detection frame and all predicted tracks in the high-score frame set to obtain a cost matrix; Based on the cost matrix, using a Hungary algorithm to perform primary matching on the vehicle detection frames in the high-score frame set and all the predicted tracks, performing Kalman filtering track synthesis on the predicted tracks successfully matched once and the vehicle detection frames, generating tracking tracks and updating the tracking tracks into the current track set; And carrying out secondary matching on the prediction track which is not successfully matched with the vehicle detection frame in the low-score frame set, carrying out Kalman filtering track synthesis on the prediction track which is successfully matched with the detection frame, generating a tracking track and updating the tracking track into the current track set.
9. The method of claim 1, wherein after estimating the speed of the vehicle target from the vehicle true trajectory, the method further comprises: And smoothing the calculated instantaneous speed by adopting an exponential smoothing moving average method to output a stable vehicle speed value.
10. A road vehicle speed estimation system based on unmanned aerial vehicle inspection, for implementing the road vehicle speed estimation method based on unmanned aerial vehicle inspection according to any one of claims 1 to 9, the system comprising: The system comprises a vehicle detection and tracking module, a vehicle track processing module and a vehicle track processing module, wherein the vehicle detection and tracking module is used for acquiring real-time video data of a road surface through an onboard camera of an unmanned aerial vehicle; The motion compensation module is used for constructing a dense optical flow field between two adjacent frames of images based on a dense optical flow algorithm, screening abnormal points of the dense optical flow field according to the motion state of the background in the vehicle detection frame and the real-time video data, removing the abnormal points to obtain an effective background pair, calculating according to the effective background pair to obtain a homography matrix, and performing motion compensation on the target track according to the homography matrix to generate a real track of the vehicle; And the adaptive speed estimation module is used for acquiring the actual diagonal length and the pixel diagonal length of the vehicle target and estimating the speed of the vehicle target according to the actual track of the vehicle, the actual diagonal length and the pixel diagonal length.

Description

Road vehicle speed estimation method and system based on unmanned aerial vehicle inspection Technical Field The application relates to the technical field of intelligent traffic and computer vision intersection, in particular to a road vehicle speed estimation method and system based on unmanned aerial vehicle inspection. Background The ground vehicle speed estimation is an important component of an intelligent traffic system, is important to the improvement of road traffic efficiency and traffic safety, and is widely applied to the fields of traffic control, emergency rescue, key area security and the like. The existing vehicle speed estimation technology mainly comprises radar speed measurement, laser speed measurement, ultrasonic speed measurement, speed measurement based on video processing and the like. However, the equipment required by radar speed measurement and laser speed measurement is usually high in price and needs frequent maintenance, the ultrasonic speed measurement is low in cost, but long in measurement response time and small in effective distance, and the speed measurement based on video processing has the advantages of convenience in installation and maintenance, abundant information, mature technology and the like, so that the method is rapidly developed. Under the general condition, a speed measuring method based on video processing is used for estimating the speed of a road vehicle, video data are collected by mainly depending on a monitoring camera fixedly erected on the road, the vehicle is identified through a target detection algorithm, the vehicle is tracked by a plurality of frames through a tracking algorithm to obtain a target track, and simultaneously, a mapping relation between a pixel coordinate system and an actual distance is built by combining a pre-calibrated camera internal parameter and an external parameter, so that the speed of the vehicle is estimated. However, the existing speed measurement method based on video processing is limited by insufficient field coverage of a camera fixedly erected, and the intensive laying of monitoring equipment is required to realize the full coverage of a road network, and complex traffic conditions such as multi-lane intensive traffic flow, vehicle cross shielding and the like cannot be dealt with. With the rapid development of civil unmanned aerial vehicle technology, the vehicle speed measurement method based on the onboard video becomes an application hot spot due to the advantages of low equipment deployment cost, wide monitoring coverage range, strong space maneuverability and the like. The existing unmanned aerial vehicle-mounted video road surface vehicle speed estimation technology utilizes a GIS system to acquire GPS coordinates of a ground reference point, and estimates the vehicle speed based on multi-frame vehicle displacement changes by establishing pixel-geographic coordinate mapping. However, the scheme is required to be provided with ground control points in advance, has high field operation complexity, is also influenced by factors such as large displacement estimation deviation caused by dynamic background interference caused by unmanned aerial vehicle motion, manual intervention required for the mapping relation between pixel dimensions and actual distances, and has poor scheme adaptability. Therefore, in a complex environment, vehicle speed estimation accuracy and real-time performance based on unmanned aerial vehicle on-board video still face challenges. Disclosure of Invention The application provides a road vehicle speed estimation method and system based on unmanned aerial vehicle inspection, which at least solve the problem of low vehicle speed estimation precision based on unmanned aerial vehicle-mounted video in the prior art. In a first aspect, the present application provides a method for estimating a speed of a road vehicle based on unmanned aerial vehicle inspection, including: Acquiring real-time video data of a road surface through an onboard camera of the unmanned aerial vehicle, and detecting a vehicle target in the real-time video data by adopting a trained target detection model to generate a vehicle detection frame and a confidence coefficient thereof; Tracking a vehicle detection frame and the confidence coefficient thereof in a frame image of the real-time video data through a multi-target tracking algorithm to obtain a target track of the vehicle target; Constructing a dense light flow field between two adjacent frames of images based on a dense light flow algorithm, screening abnormal points of the dense light flow field according to the motion states of the vehicle detection frame and the background in the real-time video data, and removing the abnormal points to obtain an effective background pair; and calculating to obtain a homography matrix according to the effective background pair, performing motion compensation on the target track according to the homography matrix, generating a rea