CN-119887945-B - High-speed depth calculation method for large scene

CN119887945BCN 119887945 BCN119887945 BCN 119887945BCN-119887945-B

Abstract

The invention discloses a high-speed depth calculation method for a large scene, which comprises the following steps of firstly calibrating a binocular camera and a depth camera respectively, determining internal parameters and external parameters of the binocular camera and the depth camera, preprocessing image data acquired by the binocular camera, rapidly scanning and analyzing the large scene in the image data by utilizing a rapid positioning algorithm so as to position a target area, mapping the positioned target area to a pixel coordinate system of the depth camera, and carrying out depth calculation on points in the target area mapped to the pixel coordinate system of the depth camera by utilizing the depth camera. According to the method, the depth information of the target area is quickly acquired in a large scene by combining a staged strategy with a quick positioning target area and a depth calculation technology.

Inventors

SUN DANFENG
LIN JINFENG
ZHAO JIANYONG
WU HUIFENG

Assignees

杭州电子科技大学

Dates

Publication Date: 20260508
Application Date: 20250102

Claims (9)

1. The high-speed depth calculation method for the large scene is characterized by comprising the following steps of: S100, calibrating a binocular camera and a depth camera respectively, and determining an internal reference matrix and an external reference matrix of the binocular camera and the depth camera; s200, preprocessing image data acquired by a binocular camera; s300, carrying out rapid scanning and analysis on a large scene in the image data by using a rapid positioning algorithm so as to position a target area; S301, extracting feature points in left and right images acquired by a binocular camera through feature point descriptors; s302, feature point matching among binocular cameras is carried out, and geometric verification is applied to remove error matching; s303, calculating world coordinates of the matched feature points by combining an internal reference matrix and an external reference matrix of the binocular camera; S304, supplementing depth information of the missing region by a bilinear interpolation method; S305, combining the depth information and the image to generate a point cloud; S306, aggregating the point clouds into a target area according to the conditions set by the user and outputting the target area; s400, mapping the positioned target area to a pixel coordinate system of a depth camera; The step S400 includes the following sub-steps: step S401, converting pixel coordinates into points in a camera coordinate system: for each point in the camera 1, an internal reference matrix is used Converting them from pixel coordinates to points in the camera coordinate system of the camera 1, the points being represented in homogeneous coordinates, the conversion formula being: ; Step S402, converting camera coordinates into points in a world coordinate system: Each point is converted from the camera coordinate system into the world coordinate system using the extrinsic matrix of the camera 1, with the conversion formula: ; step S403, converting the world coordinate system into points in the second camera coordinate system: Each point is converted from the world coordinate system to the camera coordinate system of the camera 2 using the extrinsic matrix of the camera 2, with the conversion formula: ; step S404, projecting the point in the second camera coordinate system to the pixel coordinate system of the camera 2: each point is converted from the camera coordinate system of the camera 2 to the image coordinate system of the camera 2 using the reference matrix of the camera 2, the conversion formula being: ; s500, performing depth calculation on points in a target area mapped to a pixel coordinate system of the depth camera by using the depth camera.
2. The high-speed depth computing method of claim 1, wherein the method for determining the internal reference matrix and the external reference matrix comprises the following steps: s101, selecting a set of fixed-position calibration points which can be observed in the fields of view of a binocular camera and a depth camera at the same time; s102, determining an internal reference matrix of the binocular camera by shooting an image of the space calibration point 、 And depth camera's internal reference matrix ; S103, combining the internal reference matrix by obtaining the mapping relation between each camera pixel point and the corresponding real space coordinate 、、 External parameter matrix of binocular camera , External reference matrix for depth camera 。
3. The method according to claim 2, wherein in step S103, the perspective model and the known coordinates of the calibration points are used to solve the outlier matrix.
4. A high-speed depth calculation method for a large scene according to claim 3, wherein the solution method for the extrinsic matrix is as follows: assume that in three-dimensional space, there are coordinates of four points that are not on the same plane, and their corresponding pixel points: ; from the perspective model, the following equation can be established: ; wherein, the Is a scale factor of the number of times, Is an internal reference matrix, and the reference matrix is a reference matrix, And The rotation matrix and the translation vector of the camera respectively can calculate the external parameter matrix 。
5. The high-speed depth calculation method of claim 1, wherein the preprocessing method of the image data includes denoising and gray scale processing.
6. The method according to claim 5, wherein the step S302 further comprises feature point matching between binocular cameras, wherein the feature point matching is performed by applying geometric verification, identifying and removing erroneous matching, and checking whether each pair of matching points follows corresponding geometric constraint by calculating a base matrix or an essential matrix, so as to eliminate inconsistent matching.
7. The high-speed depth computing method of claim 5, wherein in step S303, the computing method of world coordinates of feature point matching is as follows: First, let: ; converting three-dimensional depth calculation problem into solution equation Wherein the content of the vector is the world coordinates of the feature points that need ranging, noted as wherein the matrix expression is as follows: ; , ; And solving the world coordinates of the feature points in the world coordinate system by a least square method for the world coordinates of the measured points.
8. A high-speed depth calculation method according to claim 5 or 6, wherein in step S301, the selected feature descriptors have scale rotation invariance.
9. The method according to claim 1, wherein in the step S500, the depth calculation is performed using the structured light principle.

Description

High-speed depth calculation method for large scene Technical Field The invention relates to the technical field of computer vision and three-dimensional reconstruction, in particular to a high-speed depth calculation method for a large scene. Background In the fields of computer vision and image processing, the complete three-dimensional reconstruction of large scenes is a challenging task. The traditional three-dimensional reconstruction method generally needs to perform global optimization on the whole scene, and the computational complexity grows exponentially along with the increase of the scene scale, so that the problem of slower speed when processing a large scene is caused, and the instantaneity and the efficiency are limited, especially for the large scene. In addition, conventional methods may also be affected by environmental changes and occlusions, resulting in poor reconstruction accuracy or failure. In order to solve these problems, some improved methods for three-dimensional reconstruction of large scenes, such as algorithms based on parallel computing and methods based on GPU acceleration, have been proposed in recent years. However, these methods still have certain limitations, such as high requirements for hardware devices, high implementation complexity, and the like. Therefore, there is a need to develop a faster, more efficient method to solve the three-dimensional reconstruction problem of large scenes. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a high-speed depth calculation method for a large scene, and aims to solve the problem that the speed is low when the traditional method is used for processing the large scene. According to the method, the depth information of the target area is quickly acquired in a large scene by combining a staged strategy with a quick positioning target area and a depth calculation technology. In order to solve the technical problems, the technical scheme of the invention is as follows: A high-speed depth calculation method for a large scene comprises the following steps: S100, calibrating a binocular camera and a depth camera respectively, and determining an internal reference matrix and an external reference matrix of the binocular camera and the depth camera; s200, preprocessing image data acquired by a binocular camera; s300, carrying out rapid scanning and analysis on a large scene in the image data by using a rapid positioning algorithm so as to position a target area; s400, mapping the positioned target area to a pixel coordinate system of a depth camera; s500, performing depth calculation on points in a target area mapped to a pixel coordinate system of the depth camera by using the depth camera. Preferably, the method for determining the internal reference matrix and the external reference matrix comprises the following steps: s101, selecting a set of fixed-position calibration points which can be observed in the fields of view of a binocular camera and a depth camera at the same time; S102, determining an internal reference matrix K 1、K2 of the binocular camera and an internal reference matrix K 3 of the depth camera by shooting images of space calibration points; S103, by obtaining the mapping relation between each camera pixel point and the corresponding real space coordinate and combining the internal reference matrix K 1、K2、K3, the external reference matrix [ R 1|t1],[R2|t2 ] of the binocular camera and the external reference matrix [ R 3|t3 ] of the depth camera are calculated. Preferably, in the step S103, the perspective model and the known coordinates of the calibration points are used to solve the extrinsic matrix. Preferably, the solving method of the extrinsic matrix is as follows: assume that in three-dimensional space, there are coordinates of four points that are not on the same plane, and their corresponding pixel points: Pword1＝(X1,Y1,Z1),ppixel1＝(x1,y1,1) Pword2＝(X2,Y2,Z2),ppixel2＝(x2,y2,1) Pword3＝(X3,Y3,Z3),ppixel3＝(x3,y3,1) Pword4＝(X4,Y4,Z4),ppixel4＝(x4,y4,1) from the perspective model, the following equation can be established: Where s is a scale factor, K is an internal reference matrix, R and t are rotation and translation vectors of the camera, respectively, and the external reference matrix [ R|t ] can be calculated. Preferably, the preprocessing method of the image data includes denoising and gray scale processing. Preferably, in the step 300, the method for positioning the target area is as follows: S301, extracting feature points in image data acquired by a binocular camera through feature point descriptors; s302, calculating world coordinates matched with the feature points by combining an internal reference matrix and an external reference matrix of the binocular camera; s303, calculating depth information of the missing region by a bilinear interpolation method; s304, combining the depth information and the image to generate a point cloud, and aggregating the point cloud in