CN-122023484-A - Multi-source heterogeneous point cloud registration method and related device based on image key points

CN122023484ACN 122023484 ACN122023484 ACN 122023484ACN-122023484-A

Abstract

The application discloses a multi-source heterogeneous point cloud registration method and a related device based on image key points, which relate to the technical field of three-dimensional data processing, the method comprises the steps of extracting ground point cloud from target and source point cloud, determining unit normal vector and height average value, correcting the source point cloud by using a rotation matrix and a vertical translation vector, projecting the source point cloud into target and source images, extracting characteristic key points, obtaining initial matching point pairs by consistency screening of descriptor similarity and main direction, and calculating a pixel motion vector field through dense optical flow improved by a local binary pattern feature map, selecting a homonymous matching point pair according to a displacement threshold, merging and de-duplicating to obtain a potential image matching point set, establishing geometric constraint, obtaining optimal affine transformation parameters, back-projecting to obtain point cloud corresponding coordinates, solving horizontal two-dimensional rigid transformation, calculating a coarse registration optimal space transformation matrix by combining a rotation matrix and a vertical translation vector, and finally finishing multi-source heterogeneous point cloud fine registration by using an iterative closest point algorithm.

Inventors

Xu Mengbing
Hui Jike
LI HAO
MA ZHAOTING
ZHAO YUANCHUN
ZHONG XUETING
YAN PENG
FANG CHIYU
LIU DONG
YONG QI
WANG XU

Assignees

中国测绘科学研究院

Dates

Publication Date: 20260512
Application Date: 20260130

Claims (10)

1. The multi-source heterogeneous point cloud registration method based on the image key points is characterized by comprising the following steps of: Extracting ground point cloud from target point cloud and source point cloud respectively And ground point cloud And respectively determine the ground point clouds And ground point cloud Is a unit normal vector and a height average value; Correcting the source point cloud in the vertical direction based on a rotation matrix and a vertical translation matrix, wherein the rotation matrix is based on the ground point cloud And ground point cloud The vertical translation matrix is obtained by solving the unit normal vector of the plane point cloud And ground point cloud The difference of the height mean values is calculated; extracting characteristic key points in a target image and a source image, and screening initial matching point pairs based on a double constraint condition of local similarity and main direction consistency of descriptors, wherein the target image and the source image are obtained by projecting a target point cloud and a corrected source point cloud; The method is characterized in that a dense optical flow method based on the improvement of a local binary pattern feature map is used for calculating a pixel-level motion vector field and screening homonymous matching point pairs according to a displacement threshold; Performing union de-duplication on the initial matching point pair and the homonymous matching point pair to obtain a potential image matching point set; Establishing geometric consistency constraint based on the potential image matching point set, and adopting an iterative re-weighted least square optimization framework to obtain optimal affine transformation parameters for aligning the potential image matching point set; Back projecting to obtain corresponding point coordinates in the target point cloud and the source point cloud according to the image matching point pair obtained by the optimal affine transformation parameters, and solving two-dimensional rigid transformation in the horizontal direction based on the corresponding point coordinate pair in the target point cloud and the source point cloud; And after correcting the initial rotation and translational position deviation between the target point cloud and the source point cloud, adopting an iterative nearest point algorithm to finish the fine registration of the target point cloud and the source point cloud based on the rough registration optimal space transformation matrix of the source point cloud.
2. The multi-source heterogeneous point cloud registration method based on the image key points according to claim 1, wherein the two constraint conditions of the local similarity and the main direction consistency of the descriptors specifically comprise: the formula expression of the partial similarity of the descriptors is as follows: ; The formula expression of the main direction consistency is as follows: ; Wherein, the And Respectively, in the target image and The two feature keypoints with the smallest and next smallest keypoint distances, In the form of a ratio value, For the principal direction angle of the i-th keypoint in the source image, Is the principal direction angle of the jth key point in the target image.
3. The multi-source heterogeneous point cloud registration method based on the image key points according to claim 1, wherein solving the two-dimensional rigid transformation in the horizontal direction based on the corresponding point coordinate pairs in the target point cloud and the source point cloud specifically comprises: The coordinate pairs of the corresponding points in the target point cloud and the source point cloud are respectively characterized as a target point set And a source point set ; According to the formula Solving a two-dimensional rigid transformation in the horizontal direction, wherein, In the form of a two-dimensional orthogonal rotation matrix, Is a translation vector.
4. The method for registering multi-source heterogeneous point clouds based on image key points according to claim 1, wherein the local binary pattern feature map is extracted from a target image and a source image, the method for improving dense optical flow based on the local binary pattern feature map calculates a pixel-level motion vector field, and screens homonymous matching point pairs according to a displacement threshold, and the method specifically comprises the following steps: According to the formula Determining LBP value of pixel point, Is a pixel The gray value at which the color is to be changed, For the radius of the neighborhood point, For the angle of the neighborhood point, Is a sign function when Time of day Otherwise ; Determining optical flow fields for source and target images In the formula (I), in the formula (II), Is the position of the source image Optical flow at the point(s) of the flow, Vector displacements in the horizontal direction and the vertical direction are respectively represented; Calculating pixel changes between a source image and a target image based on Farneback dense optical flow estimation methods, and calculating a pixel-level motion vector field; setting threshold values based on optical flow vector displacement And eliminating the mismatching points to obtain the homonymous matching point pairs.
5. The method of image keypoint-based multi-source heterogeneous point cloud registration as claimed in claim 4, wherein the threshold is set according to an optical flow vector displacement The method for eliminating the mismatching points specifically comprises the following steps: According to the formula Removing mismatching points; In the formula, The valid homonyms that represent the image matches correspond, , Respectively representing the abscissa and the ordinate of the ith feature point in the source image; , respectively representing the actual matching coordinates of the jth feature point in the target image.
6. The multi-source heterogeneous point cloud registration method based on image key points according to claim 1, wherein the method is characterized in that geometrical consistency constraint is established based on the potential image matching point set, and an iterative re-weighted least square optimization framework is adopted to obtain optimal affine transformation parameters for aligning the potential image matching point set, and specifically comprises the following steps: Establishing geometric consistency constraint based on the potential image matching point set, and obtaining optimal affine transformation parameters for aligning the potential image matching point set by adopting an iterative re-weighted least square optimization framework combined with a Huber robust kernel function, wherein the iterative re-weighted least square optimization framework is used for inhibiting the influence of a mismatching relation caused by discrete noise or shielding missing factors on image registration precision, and the formula expression of the overall objective function combined with the Huber robust kernel function is as follows In which, in the process, In order to be a Huber cost function, For affine transformation residual , For parameter minimization, a is the linear transformation part in the affine transformation matrix and t is the translation vector.
7. The method for registering multi-source heterogeneous point clouds based on image key points according to claim 1, wherein the formula of the coarse registration optimal space transformation matrix of the source point clouds is as follows: ; In the formula, For a vertically oriented rotation matrix, For the translation matrix in the vertical direction, Is a two-dimensional orthogonal rotation matrix in the horizontal direction, Is a translation vector in the horizontal direction, 、、 To register the three-dimensional coordinates of all points in the source point cloud, 、、 Three-dimensional coordinates of all points in the source point cloud before registration.
8. A multi-source heterogeneous point cloud registration device based on image keypoints, configured to implement a multi-source heterogeneous point cloud registration method based on image keypoints as described in any one of claims 1-7, and comprising: The ground point cloud extraction module is used for extracting the ground point cloud from the target point cloud and the source point cloud respectively And ground point cloud And respectively determine the ground point clouds And ground point cloud Is a unit normal vector and a height average value; The vertical direction correction module is used for carrying out vertical direction correction on the source point cloud based on a rotation matrix and a vertical direction translation matrix, wherein the rotation matrix is based on the ground point cloud And ground point cloud The vertical translation matrix is obtained by solving the unit normal vector of the plane point cloud And ground point cloud The difference of the height mean values is calculated; The first screening module is used for extracting characteristic key points in a target image and a source image, and screening initial matching point pairs based on a double constraint condition of local similarity of descriptors and main direction consistency, wherein the target image and the source image are obtained by projecting a target point cloud and a corrected source point cloud; The second screening module is used for calculating a pixel-level motion vector field based on a dense optical flow method improved by the local binary pattern feature map and screening homonymous matching point pairs according to a displacement threshold; The potential image matching point set determining module is used for carrying out union de-duplication on the initial matching point pair and the homonymous matching point pair to obtain a potential image matching point set; The constraint module is used for establishing geometric consistency constraint based on the potential image matching point set, and obtaining optimal affine transformation parameters for aligning the potential image matching point set by adopting an iterative re-weighted least square optimization framework; The back projection module is used for obtaining corresponding point coordinates in the target point cloud and the source point cloud according to the image matching point pair obtained by the optimal affine transformation parameters by back projection, and solving two-dimensional rigid transformation in the horizontal direction based on the corresponding point coordinate pair in the target point cloud and the source point cloud; The registration module is used for calculating a rough registration optimal space transformation matrix of the source point cloud based on the rotation matrix, the vertical translation matrix and the two-dimensional rigid transformation, correcting initial rotation and translational displacement deviation between the target point cloud and the source point cloud based on the rough registration optimal space transformation matrix of the source point cloud, and completing fine registration of the target point cloud and the source point cloud by adopting an iterative nearest point algorithm.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor executes the computer program to implement a multi-source heterogeneous point cloud registration method based on image keypoints as claimed in any of claims 1-7.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a multi-source heterogeneous point cloud registration method based on image keypoints according to any of claims 1-7.

Description

Multi-source heterogeneous point cloud registration method and related device based on image key points Technical Field The application relates to the technical field of three-dimensional data processing, in particular to a multi-source heterogeneous point cloud registration method based on image key points and a related device. Background In the field of three-dimensional data processing, point cloud registration is used as a key technology for unifying point cloud data under different coordinate systems to the same coordinate system, and has important application value in various fields such as reverse engineering, three-dimensional reconstruction, automatic driving and the like. However, when the point cloud data to be registered originates from different modality sensors and different observation platforms (e.g., fixed station lidar, vehicle-mounted mobile lidar, depth camera, etc.), multi-source heterogeneous point clouds are formed. Such point clouds often have significant differences in point density, noise level, data distribution characteristics, and scale, which present a significant challenge to the registration task. The traditional point cloud registration method, such as a registration algorithm based on point features, often causes low registration efficiency, low precision and even registration failure due to difficult feature extraction, low initial matching precision or insufficient robustness when facing to multi-source heterogeneous point clouds. Particularly, under the condition of lacking good initial transformation estimation, the direct application of the fine registration algorithm such as Iterative Closest Point (ICP) and the like easily falls into a local optimal solution, and an ideal registration result is difficult to obtain. Therefore, how to realize efficient, accurate and robust registration aiming at the characteristics of multi-source heterogeneous point clouds is a problem to be solved in the technical field of current three-dimensional data processing. Disclosure of Invention The application aims to provide a multi-source heterogeneous point cloud registration method and a related device based on image key points, wherein the registration process is divided into two stages of vertical direction alignment based on ground points and horizontal direction alignment based on image feature matching, and a multi-constraint geometric feature matching strategy is constructed to realize high-precision registration so as to improve the precision and robustness of heterogeneous platform point cloud registration. In order to achieve the above object, the present application provides the following solutions: in a first aspect, the present application provides a multi-source heterogeneous point cloud registration method based on image key points, including: Extracting ground point cloud from target point cloud and source point cloud respectively And ground point cloudAnd respectively determine the ground point cloudsAnd ground point cloudIs a unit normal vector and a height average value; Correcting the source point cloud in the vertical direction based on a rotation matrix and a vertical translation matrix, wherein the rotation matrix is based on the ground point cloud And ground point cloudThe vertical translation matrix is obtained by solving the unit normal vector of the plane point cloudAnd ground point cloudThe difference of the height mean values is calculated; extracting characteristic key points in a target image and a source image, and screening initial matching point pairs based on a double constraint condition of local similarity and main direction consistency of descriptors, wherein the target image and the source image are obtained by projecting a target point cloud and a corrected source point cloud; The method is characterized in that a dense optical flow method based on the improvement of a local binary pattern feature map is used for calculating a pixel-level motion vector field and screening homonymous matching point pairs according to a displacement threshold; Performing union de-duplication on the initial matching point pair and the homonymous matching point pair to obtain a potential image matching point set; Establishing geometric consistency constraint based on the potential image matching point set, and adopting an iterative re-weighted least square optimization framework to obtain optimal affine transformation parameters for aligning the potential image matching point set; Back projecting to obtain corresponding point coordinates in the target point cloud and the source point cloud according to the image matching point pair obtained by the optimal affine transformation parameters, and solving two-dimensional rigid transformation in the horizontal direction based on the corresponding point coordinate pair in the target point cloud and the source point cloud; And after correcting the initial rotation and translational position deviation between the target point