CN-116309702-B - Intelligent vehicle target tracking system and method based on monocular camera

Abstract

The invention discloses an intelligent vehicle target tracking system and method based on a monocular camera. The system includes a sensor group and a target tracker. The sensor group includes monocular vision, GNSS and INS sensors. The target tracker comprises monocular vision target detection, communication, vehicle pose detection and target tracking modules. The monocular vision target detection module acquires target azimuth information from the monocular vision sensor, the communication module acquires GNSS differential information from the positioning base station, the vehicle pose detection module reads the measurement data of the GNSS and INS sensors and the GNSS differential information to calculate vehicle pose information, and the target tracking module performs target tracking by utilizing the vehicle maneuvering information and the target azimuth information calculated by the vehicle pose information to obtain the movement state information such as azimuth, distance, speed and the like of the target. The invention improves the target tracking precision of the monocular camera of the intelligent vehicle, particularly the target tracking precision in a ramp scene, and can effectively improve the reliability and safety of an auxiliary driving and automatic driving system.

Inventors

• CEN MING
• WEN HENGCONG
• ZHANG JING
• ZENG SUHUA

Assignees

• 重庆邮电大学

Dates

Publication Date

20260512

Application Date

20230216

Claims (1)

1. 1. An intelligent vehicle target tracking system based on a monocular camera is characterized by comprising a sensor group and a target tracker, wherein: The sensor group comprises a monocular vision sensor, a Global Navigation Satellite System (GNSS) sensor and an Inertial Navigation System (INS) sensor, wherein the monocular vision sensor is used for acquiring and outputting external environment image data, the GNSS sensor is used for acquiring and outputting longitude and latitude information and course angle information of the intelligent vehicle, and the INS sensor is used for acquiring and outputting acceleration information and angular velocity information of the course angle of the intelligent vehicle; The target tracker comprises a monocular vision target detection module, a communication module, a vehicle pose detection module and a target tracking module; the monocular vision target detection module acquires target azimuth information from the image; the vehicle pose detection module reads the longitude and latitude and course angle information of the intelligent vehicle, the angular velocity information of the intelligent vehicle, the course angle and the position pose information of the vehicle, which are acquired by the GNSS sensor, and the angular velocity information of the intelligent vehicle, which are acquired by the INS sensor; The target tracking process of the target tracker comprises the following steps: 2.1 target orientation detection the monocular visual target detection module obtains the original image information from the monocular visual sensor and calculates to obtain the target orientation measurement set under the vehicle coordinate system Wherein n is the number of target measurements detected by the monocular vision sensor at time k; 2.2, obtaining differential correction amount, namely obtaining GNSS differential information by a communication module and sending the GNSS differential information to a vehicle pose detection module; 2.3 calculating the pose information of the vehicle, namely, a vehicle pose detection module reads the longitude, latitude and course angle information of the intelligent vehicle, which are acquired by a GNSS sensor, the angular velocity information of the acceleration and course angle of the intelligent vehicle, which are acquired by an INS sensor, and GNSS differential information of a communication module, and calculates the pose information of the vehicle in a coordinate system fixedly connected with the ground Including the X-direction position Position in Y direction Speed in X direction Velocity in Y direction Acceleration in X direction Acceleration in Y direction Course angle Angular velocity of course angle ; 2.4 Tracking the target State, the target tracking module measures the set according to the target azimuth of the monocular vision target detection module And vehicle self pose information through a vehicle pose detection module Calculated vehicle maneuver information Tracking the target to obtain the azimuth, distance and speed movement state information of the target in the vehicle coordinate system; The step 2.4 target tracking method comprises the following steps: 3.1 The coordinate system is established by selecting a point on the ground as an origin, taking the north direction as the Y axis and the east direction as the X axis, and setting the Z axis perpendicular to the ground to establish a coordinate system fixedly connected with the ground Taking the center of the right front of the vehicle as an origin, taking the right front as a Y axis, taking the right as an X axis, and establishing a vehicle coordinate system with the Z axis perpendicular to the ground ; 3.2 Establishing a target motion state model and a measurement model; At time k, vehicle coordinate system The following target motion state model is: (1) wherein in the vehicle coordinate system The lower part of the upper part is provided with a lower part, The X-direction position, the Y-direction position, the X-direction speed and the Y-direction speed of the target at the moment k are represented; for the purpose of the target noise gain, For the target process noise at time k-1, Representing a state transition matrix of motion of the object, Representing the motion state of the target at the moment k-1; for the vehicle's own state transition matrix, Representing the time k of the vehicle relative to the coordinate system of the vehicle Lower X-direction position maneuver, Y-direction position maneuver, X-direction speed variation and Y-direction speed variation; At time k, vehicle coordinate system The lower target measurement model is: (2) Wherein, the The noise is measured by the target at the moment k; 3.3 Predicting the motion state of the target according to the motion state equation of the target in the step 3.2 and estimating the state of the target at the previous moment And covariance matrix Calculating one-step predicted values of target states respectively Covariance matrix of one-step prediction error ; 3.4 Measurement screening by Calculating a predicted observation of the target according to the metrology model A sector tracking gate is arranged to collect target measurements Screening to obtain target candidate measurement set ; 3.5 Data correlation, predicting value for target track And a target candidate measurement set Carrying out data association; for the measurement targets which are not associated, as possible new target tracks, in the subsequent n tracking processes, if the number of times of successful association is greater than a threshold C_new, the measurement targets are taken as real target tracks; as for the target track which is not associated with the target, as the target track which possibly needs to be destroyed, in the subsequent n tracking processes, if the association failure times are greater than a threshold value C_disp, destroying the track; 3.6 filtering the target state, namely filtering the associated target by using a Kalman filter to obtain the optimal estimation of the motion state of the target at the moment k And ; Step 3.2 vehicle maneuver information The calculation method comprises the following steps: 4.1 calculating vehicle pose information, namely, a vehicle pose detection module reads longitude, latitude and course angle information of an intelligent vehicle, acquired by a GNSS sensor, angular velocity information of acceleration and course angle of the intelligent vehicle, acquired by an INS sensor, and GNSS differential information of a communication module, and then calculates to obtain a coordinate system Lower vehicle pose information Including the X-direction position Position in Y direction Speed in X direction Velocity in Y direction Acceleration in X direction Acceleration in Y direction Course angle Angular velocity of course angle ; 4.2 Calculating vehicle maneuver information according to the coordinate System in 4.1 Lower vehicle pose information Vehicle pose information at previous time Calculating the vehicle maneuver information Wherein Representing the time k of the vehicle relative to the coordinate system of the vehicle Lower X-direction position maneuver, Y-direction position maneuver, X-direction speed variation, Y-direction speed variation.

Description

Intelligent vehicle target tracking system and method based on monocular camera Technical Field The invention belongs to the technical field of intelligent vehicle environment sensing, and particularly relates to an intelligent vehicle target tracking system and method based on a monocular camera. Background The intelligent vehicle is a comprehensive system integrating the functions of environment sensing, dynamic planning decision-making, multi-level auxiliary driving and the like. The environment sensing system detects the targets of the surrounding environment through vehicle-mounted sensors, such as a laser radar, a camera and a millimeter wave radar, and provides reliable basis for decision control of the vehicle. The monocular camera has lower cost and is widely applied to intelligent vehicle environment sensing systems. How to utilize the monocular camera to carry out effective target detection, the environment perception ability of improvement intelligent vehicle has important meaning and practical value. The Chinese patent application discloses a target tracking method based on monocular vision, terminal equipment and a storage medium (application number: CN 202210548990.7), wherein the method is used for tracking the distance between a target and a host vehicle and measuring the distance depending on the type of the target. The Chinese patent application discloses a monocular vision ranging method (application number: CN 201910029050.5), which aims at indoor and outdoor static environments, compensates object distance according to an image distance error formula and an incidence error formula of a monocular camera, and obtains a target distance. The Chinese patent application is a monocular vision distance measurement processing method (application number: CN 202110092093.5), which is used for preprocessing an acquired image, carrying out preliminary denoising and filtering processing on the image by adopting a Gaussian filter, and then calculating the actual distance of a target. The intelligent vehicle running environment is complex, a ramp scene often appears, and the method has the defect in the aspect of processing target detection under the ramp scene. Aiming at the complexity of the intelligent vehicle running environment, particularly in a ramp scene, the problem that a large error exists in the target detection distance of the monocular vision system, the method and the device utilize the more accurate azimuth information of the target and the maneuvering information of the intelligent vehicle to track the target, estimate the relative movement of the target and the intelligent vehicle, improve the tracking precision of the target, further improve the tracking precision of the monocular camera of the intelligent vehicle in the ramp scene, and improve the reliability and safety of auxiliary driving or automatic driving. Disclosure of Invention The present invention is directed to solving the above problems of the prior art. An intelligent vehicle target tracking system and method based on a monocular camera are provided. The technical scheme of the invention is as follows: an intelligent vehicle target tracking system based on a monocular camera, comprising a sensor group and a target tracker, wherein: The sensor group comprises a monocular vision sensor, a Global Navigation Satellite System (GNSS) sensor and an Inertial Navigation System (INS) sensor, wherein the monocular vision sensor is used for acquiring and outputting external environment image data, the GNSS sensor is used for acquiring and outputting longitude and latitude information and course angle information of the intelligent vehicle, and the INS sensor is used for acquiring and outputting acceleration information and angular velocity information of the course angle of the intelligent vehicle; The target tracker comprises a monocular vision target detection module, a communication module, a vehicle pose detection module and a target tracking module, wherein the monocular vision target detection module is used for acquiring target azimuth information from an image, the communication module is used for acquiring GNSS differential information and sending the GNSS differential information to the vehicle pose detection module, the vehicle pose detection module is used for reading intelligent vehicle longitude and latitude and course angle information acquired by a GNSS sensor, intelligent vehicle acceleration and course angle angular velocity information acquired by an INS sensor and GNSS differential information to calculate vehicle pose information, and the target tracking module is used for carrying out target tracking by utilizing vehicle maneuvering information and target azimuth information obtained by calculating the vehicle pose information to obtain movement state information including the azimuth, distance and speed of a target. A target tracking method based on the system, comprising the steps of: 2.1 target orientation