CN-121999010-A - Moving target track tracking and physical quantity measuring method based on binocular vision

CN121999010ACN 121999010 ACN121999010 ACN 121999010ACN-121999010-A

Abstract

The invention provides a moving target track tracking and physical quantity measuring method based on binocular vision, which belongs to the technical field of computer vision and image processing, and comprises the steps of acquiring a stereoscopic image sequence of a moving target acquired by a binocular camera; in the initial frame of the sequence, a moving target is identified through a target detection model and an initial position is determined, a tracking query vector is generated based on the position, a continuous target area is output by tracking the moving target in the subsequent frame according to the vector, a left-right correction image is obtained by correcting each frame of image lines in the stereoscopic image sequence based on a pre-calibrated camera parameter, parallax data is obtained by calculating parallax in the continuous target area based on the image, a moving track of the moving target is obtained according to the parallax data and the calibration parameter, and a moving physical quantity is calculated according to the moving track. The invention realizes the real-time measurement and calculation of the target track and the physical quantity in the motion scene by optimizing the whole flow of detection tracking, binocular vision processing and physical quantity measurement and calculation.

Inventors

ZHANG WEIGANG
QI ZHAOBO
ZHONG JIANPING
LEI JUNJIE

Assignees

哈尔滨工业大学（威海）

Dates

Publication Date: 20260508
Application Date: 20260410

Claims (10)

1. The method for tracking the moving target track and measuring the physical quantity based on binocular vision is characterized by comprising the following steps of: acquiring a stereoscopic image sequence about a moving object acquired by a binocular camera; generating a tracking query vector based on the initial position, tracking the moving target in a subsequent frame according to the tracking query vector, and outputting a boundary frame of the moving target in an image coordinate system frame by frame as a continuous target area; Carrying out polar line correction on each frame of stereoscopic image in the stereoscopic image sequence based on pre-calibrated binocular camera parameters to obtain left and right correction images with polar lines aligned; Obtaining a motion track of the moving target in a three-dimensional space according to the parallax data and calibration parameters of the binocular camera; and calculating the motion physical quantity of the moving object according to the motion trail.
2. The binocular vision-based moving object trajectory tracking and physical quantity measuring method of claim 1, further comprising, after acquiring the sequence of stereoscopic images about the moving object acquired by the binocular camera: Preprocessing a stereoscopic image sequence, acquiring a frame rate and a frame height from meta information of the stereoscopic image sequence, and sampling an original image frame in a time dimension at a preset sampling rate to obtain a sampled image frame.
3. The binocular vision-based moving object trajectory tracking and physical quantity measuring method of claim 2, wherein in a start frame of the stereoscopic image sequence, a moving object is identified by an object detection model and an initial position is determined, a tracking query vector is generated based on the initial position, the moving object is tracked in a subsequent frame according to the tracking query vector, and a bounding box of the moving object under an image coordinate system is output frame by frame as a continuous object region, specifically: outputting an initial boundary frame of the moving object through the object detection model, wherein the initial boundary frame is characterized by the center point coordinates of the initial boundary frame and the size of the initial boundary frame; generating a tracking query vector based on the center point coordinates of the initial boundary box, wherein the tracking query vector comprises the initial position of the moving target in the initial frame and the initial frame identification; Inputting the tracking query vector into a tracking model, wherein the tracking model tracks a moving target frame by frame in a subsequent frame of a stereoscopic image sequence according to the tracking query vector, and acquires a position vector of a tracking point; And outputting a tracking boundary frame of the moving object under the image coordinate system frame by frame according to the position vector, wherein the tracking boundary frame is characterized by the center point coordinates of the tracking boundary frame and the size of the tracking boundary frame, and taking the tracking boundary frame output frame by frame as a continuous target area of the moving object.
4. The binocular vision-based moving object trajectory tracking and physical quantity measuring method of claim 1, wherein epipolar correction is performed on each frame of stereoscopic image in the stereoscopic image sequence based on a pre-calibrated binocular camera parameter to obtain epipolar-aligned left and right corrected images, specifically: Based on distortion coefficients of the binocular camera, respectively carrying out distortion elimination on left and right images in the stereoscopic image to obtain undistorted left and right images, wherein the distortion coefficients comprise radial distortion coefficients And tangential distortion coefficient Wherein, the method comprises the steps of, Representing a second order radial distortion coefficient; representing a fourth order radial distortion coefficient; Representing a sixth order radial distortion coefficient; representing a first tangential distortion coefficient; representing a second tangential distortion coefficient; Based on internal parameters and external parameters of the binocular camera, carrying out three-dimensional correction on undistorted left and right images to obtain left and right corrected images with polar alignment, wherein the internal parameters comprise focal lengths of left and right cameras And principal point coordinates The external parameters comprise a rotation matrix Translation vector 。
5. The binocular vision based moving object trajectory tracking and physical quantity measuring method of claim 4, wherein the distortion removing process is as follows: For any pixel point in the left and right images, setting the pixel coordinates in the original distorted image as Converting its pixel coordinates into normalized image coordinates with principal point as origin and focal length as unit ; Calculating coordinates after radial distortion correction according to the radial distortion coefficient Expressed as: ; Wherein, the ; The distance from any pixel point to the origin point is set; Calculating normalized coordinates after tangential distortion correction according to tangential distortion coefficients Expressed as: ; According to internal reference, correcting the tangential distortion to obtain normalized coordinates Conversion to corrected pixel coordinates Expressed as: ; Wherein, the Respectively cameras are at Shaft and method for producing the same A pixel focal length in an axial direction; Is the principal point coordinates; according to the corrected pixel coordinates From original distorted images by interpolation And (3) sampling pixel values, and reconstructing an undistorted image.
6. The binocular vision-based moving object trajectory tracking and physical quantity measuring method of claim 5, wherein the process of the stereoscopic correction is: based on a rotation matrix Translation vector Calculating correction rotation matrix of left and right images 、 And a corrected projection matrix 、 Depth mapping matrix ; Correction rotation matrix based on left and right images 、 And projection matrix 、 Generating a pixel mapping table of the left image and the right image by combining the internal references; And resampling the undistorted left and right images according to the pixel mapping table to obtain left and right corrected images with aligned polar lines, wherein corresponding pixel points of the left and right images in the left and right corrected images are positioned on the same horizontal scanning line.
7. The binocular vision-based moving object trajectory tracking and physical quantity measuring method according to claim 1, wherein parallax is calculated in the continuous object region based on the left and right corrected images to obtain parallax data corresponding to a moving object, specifically: performing horizontal Sobel operator edge enhancement processing on partial images corresponding to the continuous target area in the left and right corrected images; On the local image after edge enhancement, calculating initial parallax by adopting a semi-global block matching algorithm, wherein the size of a matching cost calculation window of the algorithm is a preset window size, and a smoothing penalty coefficient is set as And And (3) Dynamically planning and accumulating matching cost along a plurality of preset directions, and selecting the parallax with the minimum total cost as the initial parallax of the pixel; performing left-right consistency check, uniqueness check and mismatching rejection on the initial parallax to generate optimized parallax data corresponding to the moving target 。
8. The binocular vision-based moving object trajectory tracking and physical quantity measuring method of claim 7, wherein the moving object trajectory in the three-dimensional space is obtained according to the parallax data and calibration parameters of the binocular camera, specifically: From parallax data Baseline distance from binocular camera Calculating depth information of each point on the moving object Wherein The method meets the following conditions: ; Wherein, the Is the equivalent focal length of the camera; representing the parallax offset; Combining the depth information Coordinates of moving objects in image coordinate system Converts it into three-dimensional coordinates in the camera coordinate system The conversion relation satisfies: ; Wherein, the Representing principal point coordinates; Representing the pixel focal length; Three-dimensional coordinates to be acquired frame by frame And the motion tracks of the moving object in the three-dimensional space are formed by connecting the motion tracks in time sequence.
9. The binocular vision-based moving object trajectory tracking and physical quantity measuring method of claim 8, wherein the moving physical quantity of the moving object is calculated according to the moving trajectory, in particular, a pre-calibrated reference height of the center of the binocular camera with respect to a reference plane is obtained And extracting the three-dimensional coordinates of the moving target center point from the moving track Calculating the height of the moving object , The method meets the following conditions: ; Wherein, the A coordinate component representing a vertical direction under a camera coordinate system, the downward direction being a positive direction; extracting the vertical direction coordinates of the center point of the moving object of two continuous frames from the moving track And , Acquiring time intervals between adjacent frames Wherein , The frame rate is acquired for a binocular camera; calculating the rising speed of a moving object , The method meets the following conditions: ; the height calculated frame by frame And a rising speed And generating a height change curve and a speed change curve of the moving object according to the time sequence.
10. A moving object trajectory tracking and physical quantity measuring apparatus based on binocular vision, comprising: a binocular camera for acquiring a sequence of stereoscopic images on a moving object; A memory for storing executable instructions; A processor, respectively connected to the binocular camera and the memory, for executing executable instructions stored in the memory to implement a binocular vision-based moving object trajectory tracking and physical quantity measuring method as set forth in any one of claims 1 to 9.

Description

Moving target track tracking and physical quantity measuring method based on binocular vision Technical Field The invention belongs to the technical field of computer vision and image processing, and particularly relates to a moving target track tracking and physical quantity measuring method based on binocular vision. Background The application of the computer vision technology in body building and physical training has wide prospect. In various sports, physical quantities such as height, motion track, speed and the like of a moving object (such as barbells, balls, joints of a human body and the like) are core indexes for measuring motion normalization and strength output efficiency of athletes, and measurement accuracy and instantaneity of the physical quantities directly influence optimization of training effects and evaluation quality of athletic performances. The existing moving target physical quantity measuring technology mainly comprises three types, namely a first type of manual measuring mode, a second type of measuring technology based on two-dimensional vision, wherein the first type of measuring technology is a measuring mode which is greatly influenced by naked eyes of a coach or manually recorded by means of a simple tool, the measuring accuracy is low, continuous recording of a dynamic process cannot be achieved, instantaneous physical quantity in rapid movement is difficult to capture, the second type of measuring technology based on two-dimensional vision is used for acquiring images through a monocular camera and estimating the physical quantity through a target detection algorithm, but depth information is lacking, the measuring technology is easily influenced by shooting angles and background interference, measuring errors are large, high-accuracy training requirements cannot be met, and the third type of measuring technology based on binocular vision is used for acquiring three-dimensional space information through the parallax principle of a binocular camera, and compared with monocular vision, the measuring technology has higher measuring accuracy and is gradually applied to three-dimensional measurement of a moving scene. However, the conventional binocular vision measurement technology still has a plurality of defects in application of sports training scenes, namely, the frame rate and the resolution of the universal binocular camera are difficult to be compatible, the real-time tracking requirement of a fast moving object cannot be adapted, and track loss or measurement delay easily occurs. In physical training, a moving target often has high-speed movement characteristics, and high requirements are placed on the acquisition frame rate and the image resolution of a camera. Second, stereo matching is a core component of a binocular vision system, and optimizing stereo matching algorithm design has a decisive role in improving the accuracy of three-dimensional target recognition. The existing parallax calculation algorithm mostly adopts a traditional local matching or global matching algorithm, the local matching algorithm is easy to be subjected to mismatching in a sparse area of a target surface texture, the global matching algorithm has high calculation complexity, cannot meet the real-time requirement, and has poor robustness under the scenes of body shielding, complex background and the like of athletes. Thirdly, the camera calibration and image correction flow lacks optimization for a motion scene, so that calibration errors are accumulated, and the accuracy of depth mapping and physical quantity measurement is affected. The camera deployment position and shooting angle in the moving scene are changeable, and the conventional calibration method is difficult to adapt to the precision requirements in different scenes. Fourth, the prior art does not form a full-flow collaborative optimization scheme of 'detection tracking-parallax calculation-depth mapping-physical quantity measurement', each module operates independently and is easy to generate data faults, and the stability and measurement accuracy of the whole system are reduced. The detection tracking result cannot effectively guide the range optimization of parallax calculation, and the parallax calculation result cannot be used for feeding back the tracking module so as to improve the robustness of the shielding scene. Therefore, a technology for adapting to complex sports scenes, and integrating real-time performance and high precision of moving object detection tracking and physical quantity calculation is needed to solve the defects of the prior art and meet the requirements of professional sports training and evaluation. Disclosure of Invention In order to solve the technical problems, the invention provides a moving target track tracking and physical quantity measuring method based on binocular vision. High-precision tracking of a target track and real-time measurement of key physical quantity under a motion scene are