CN-122023765-A - Target detection system, method, equipment and medium based on unmanned aerial vehicle airborne scene

CN122023765ACN 122023765 ACN122023765 ACN 122023765ACN-122023765-A

Abstract

The invention discloses a target detection system, a method, equipment and a medium based on an unmanned aerial vehicle airborne scene, which relate to the technical field of target monitoring and solve the technical problems of low computational power and slow response caused by mismatching of a target detection model and hardware deployment, and the technical scheme is characterized in that an image transmission link is optimized at a hardware level, the delay of a video stream is reduced to within 50ms, the response delay of a cradle head is reduced to within 30ms, and the problems of current interruption and slow response are solved; the software layer performs layering lightweight on the target detection model, the accuracy is reserved while the calculation force is greatly saved, the multithreading frame skip detection is adopted to avoid buffer memory blocking, the control logic is optimized, the TensorRT acceleration is combined, the target detection rate reaches 19.2FPS, the high accuracy is maintained, the onboard calculation force constraint is adapted, and the real-time performance and the reliability of the onboard target monitoring of the unmanned aerial vehicle are improved.

Inventors

HE JIAPENG
JIA LINSHENG
WANG NING
LIANG BONING
YANG YAPENG
DU QINGBO
LIU LIYE
Dong Zhuben
LV QI
ZHAO XIAOYU
TANG YUYAO
YU HAO
LIU ZHE
FENG ZONGYANG

Assignees

中国辐射防护研究院

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (10)

1. The target detection system based on the unmanned aerial vehicle-mounted scene is characterized by comprising a control system and an unmanned aerial vehicle image acquisition system, wherein the unmanned aerial vehicle image acquisition system comprises a flight control module and a cradle head module, a camera is arranged on the cradle head module, the camera is connected with the control system through a converter, the control system is connected with the flight control module, and a protection resistor is connected in series between the control system and the cradle head module; The control system is provided with a target detection model, and the image acquired by the camera is detected through the target detection model; the control system controls the flight control module and the cradle head module through a feedback control algorithm; After receiving the video stream collected by the camera, the control system caches the video stream through a cache updating thread, caches the video stream through an image processing thread, reads the video stream and sends the video stream into a target detection model for target detection; the feedback control algorithm is expressed as: ; Wherein, the Representation of The desired output of the moment in time, Representation of The time of day visualizes the difference between the result and the desired output, Representation of A difference between the time visualization result and the desired output; Representing a time interval; Representing a saturation function; representing PID proportionality coefficients; Representing PID integral coefficients; representing PID differential coefficients; Representation of The amount of time desired to actual deviation; representing horizontal deviation, namely unmanned plane direction angle deviation; representing the focal length of the camera, The width of the image is represented and, Representing the distance between the drone and the target, The height of the image is indicated and, Representing the position coordinates of the center of the target frame in the image coordinate system; representing vertical deviation, namely pitch angle deviation of the cradle head module; Indicating the deviation of the longitudinal distance, Representing the deviation of the lateral distance, And The method is used for describing unmanned aerial vehicle flight speed deviation; Representing the distance-velocity mapping scaling-factor, Representing the desired area of the target frame, The area of the target frame is indicated, Indicating the direction angle of the unmanned plane.
2. The target detection system of claim 1, wherein the feedback control algorithm performs multi-target tracking adaptation based on target scores, and controls initiation of target switching when a new target score exceeds a threshold of a current target score, the target score being expressed as: ; Wherein, the Representing a target score, the higher the target priority; The confidence weight is represented as a weight of the confidence, Indicating the confidence of the target detection, The scale weights are represented as such, Indicating the temporal weight of the object being continuously detected, Representing the number of frames that the target is continuously detected, 。
3. The object detection system of claim 2, wherein the thread priority of the object detection system is set to be visual deviation calculation thread > unmanned aerial vehicle PID control thread > flight control module IMU data acquisition thread; The visual deviation calculation thread is used for processing the video stream acquired by the camera, and the visual deviation calculation thread and the flight control module IMU data acquisition thread are used as inputs of the unmanned aerial vehicle PID control thread.
4. The target detection system of claim 3, wherein after the target detection model obtains the target coordinates, the control module calculates a relative position of the target and the unmanned aerial vehicle according to the target coordinates, calculates a flight speed of the unmanned aerial vehicle, a direction angle of the unmanned aerial vehicle, and a pitch angle of the pan-tilt module according to the relative position, and the feedback control algorithm controls the flight control module and the pan-tilt module according to the flight speed of the unmanned aerial vehicle, the direction angle of the unmanned aerial vehicle, and the pitch angle of the pan-tilt module.
5. The object detection system of claim 4, wherein the buffering of the video stream by the cache update thread comprises: continuously reading the current latest frame stored in the opencv library cache queue through RTSCapture threads; Judging whether the buffer memory is blocked or not, if yes, replacing the first frame of the buffer memory queue, and if not, taking the first frame of the buffer memory queue, namely, the current latest frame, and continuously reading; the caching and reading the video stream by the image processing thread comprises the following steps: Caching the received video stream through an opencv library; and sequentially reading the cached video streams, and sequentially inputting the video streams into the target detection model.
6. The object detection system of claim 5, wherein the object detection model is a YOLOv s trim model, and wherein the YOLOv s trim model construction process includes: screening convolution layers with the weight absolute value smaller than a preset threshold value in a backbone network of an original YOLOv s model to obtain a redundant channel; pre-training the weight of the original YOLOv s model through the COCO data set, and performing sparse training on the VisDrone2019 data set to enable the weight of the redundant channel to approach 0, so as to obtain a first basic YOLOv s model; Removing channels with the weight absolute value smaller than a preset threshold value in the backbone network of the first foundation YOLOv s model through a batch pruning tool to obtain a pruned second foundation YOLOv s model; Embedding an ECA attention mechanism module at the output end of each CSP module of the second foundation YOLOv s model to obtain a third foundation YOLOv s model; And adjusting the precision of the third foundation YOLOv s model to obtain the YOLOv s fine-tuning model.
7. The object detection system of claim 6, wherein the YOLOv s fine tuning model training process comprises: The preprocessing process comprises the steps of performing layered freezing on the YOLOv s fine tuning model, wherein the bottom CSP module is totally frozen, the middle CSP module is frozen for 70% weight, and the upper CSP module and the ECA attention mechanism module thereof are not frozen; the training process comprises the steps of carrying out distillation training on a high layer and updating the weight of the high layer, thawing and fine tuning the middle layer after the distillation training of the high layer is finished, and obtaining a fine tuning model YOLOv s after training; And (3) precision quantization and acceleration, namely performing precision quantization and acceleration on the YOLOv s fine tuning model after training to obtain a final YOLOv5s fine tuning model.
8. An object detection method based on an unmanned aerial vehicle on-board scene, the object detection method being implemented by the object detection system according to any one of claims 1 to 7, comprising: the unmanned aerial vehicle image acquisition system acquires a target image through a camera, and the camera transmits the target image to the control system through a converter; the control system detects the target image through a deployed target detection model, and controls the flight control module and the cradle head module through a feedback control algorithm based on an image detection result.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, realizes the steps of the method for target detection based on unmanned aerial vehicle on-board scene as claimed in claim 8.
10. A computer storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the method for detecting an object based on an on-board scene of an unmanned aerial vehicle according to any one of claims 8.

Description

Target detection system, method, equipment and medium based on unmanned aerial vehicle airborne scene Technical Field The application relates to the technical field of target monitoring, in particular to a target detection system, method, equipment and medium based on an unmanned aerial vehicle airborne scene. Background The current unmanned aerial vehicle airborne scene target monitoring technology mainly surrounds algorithm model selection and deployment optimization and expansion. The method is characterized in that target detection algorithm standard models such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) are directly adopted in the algorithm, the target detection algorithm standard models are not customized for the onboard computing power, embedded onboard computers such as Jetson TX2 are deployed as carriers, the flow is mostly pre-training model transplanting and basic environment configuration, a hardware architecture is not deeply adapted, the optimization strategy is single in the practical application level, software only adjusts training parameters, hardware depends on default setting, redundant flows are reserved, and computing power utilization is affected. The prior art has the defects of mismatching of a model and onboard computing power, insufficient reasoning speed of Jetson TX2 deployed by a traditional target detection algorithm such as FASTER RCNN, SSD and the like, insufficient instantaneity, poor model adaptation, low detection precision of a model trained by a general data set on small targets of an onboard scene, complex background targets and the like, weak robustness, single deployment and optimization, no specific optimization of hardware, no light weight processing of the model, low computing power utilization rate, high system delay, easy flow of image transmission, long control link of a cradle head, and blocked video reading cache, and influence on monitoring and control coordination. The Chinese patent application with publication number of CN115063704A discloses a classification method of unmanned aerial vehicle monitoring targets, a convolutional neural network with a coder-decoder structure is constructed, a MobilenetV network is used as a backbone network, the model operation efficiency is improved to a certain extent, however, the method does not consider actual deployment, only carries out algorithm verification by using a public data set, and does not carry out actual deployment experiments. The Chinese patent application with publication number of CN108022255A discloses an unmanned aerial vehicle automatic tracking method which mainly comprises the steps of extracting a reference position and calculating a target distance, and can follow a target under the condition of no mechanical holder, but the method also does not mention actual deployment effect and does not carry out practical application test. Therefore, how to combine the object detection model with hardware deployment remains to be addressed by the present application. Disclosure of Invention The application provides a target detection system, a method, equipment and a medium based on an Unmanned Aerial Vehicle (UAV) airborne scene, which are used for combining a target detection model with hardware deployment, realizing light weight of the model, improving target detection precision, enabling the target detection model to be deeply matched with a hardware architecture, and reducing system delay. The technical aim of the application is realized by the following technical scheme: The target detection system based on the unmanned aerial vehicle-mounted scene comprises a control system and an unmanned aerial vehicle image acquisition system, wherein the unmanned aerial vehicle image acquisition system comprises a flight control module and a cradle head module, a camera is arranged on the cradle head module, the camera is connected with the control system through a converter, the control system is connected with the flight control module, and a protection resistor is connected in series between the control system and the cradle head module; The control system is provided with a target detection model, and the image acquired by the camera is detected through the target detection model; the control system controls the flight control module and the cradle head module through a feedback control algorithm; After receiving the video stream collected by the camera, the control system caches the video stream through a cache updating thread, caches the video stream through an image processing thread, reads the video stream and sends the video stream into a target detection model for target detection; the feedback control algorithm is expressed as: ; Wherein, the Representation ofThe desired output of the moment in time,Representation ofThe time of day visualizes the difference between the result and the desired output,Representation ofA difference between the time visualization result and the d