CN-120931733-B - Automatic calibration method and calibration device for laser radar-camera external parameters

CN120931733BCN 120931733 BCN120931733 BCN 120931733BCN-120931733-B

Abstract

The invention relates to the technical field of multi-sensor fusion, in particular to an automatic calibration method and a calibration device for external parameters of a laser radar-camera, wherein the method comprises the steps of synchronously acquiring point cloud data and image data of a road scene based on a vehicle-mounted laser radar and the camera; the method comprises the steps of carrying out semantic segmentation on image data to generate a semantic mask set of each image, extracting geometric attributes of a road scene from a vectorized map based on point cloud data, combining the semantic mask set, projecting a laser radar coordinate system of a vehicle-mounted laser radar and points in the vectorized map into the image to construct a target consistency function, and adjusting an external parameter matrix based on the target consistency function until the target consistency function reaches the maximum value to complete automatic calibration. Therefore, the problems that the related technology depends on a large amount of marking data or specific environments, has poor generalization capability, cannot combine a high-precision map with road prior information, causes insufficient calibration precision, and is difficult to meet the high-precision sensing requirement of automatic driving are solved.

Inventors

LI YIKANG
ZHOU JIAN
XIE FUXIN
QU FANGCHENG
DUAN CONG
ZHANG HONGPING

Assignees

武汉大学

Dates

Publication Date: 20260512
Application Date: 20250715

Claims (7)

1. An automatic calibration method of laser radar-camera external parameters is characterized by comprising the following steps: based on the vehicle-mounted laser radar and the camera, synchronously acquiring point cloud data and image data of a road scene; performing semantic segmentation on the image data to generate a semantic mask set of each image; extracting geometric attributes of the road scene from a vectorized map based on the point cloud data, and projecting a laser radar coordinate system of the vehicle-mounted laser radar and points in the vectorized map into an image in combination with the semantic mask set to construct a target consistency function; Adjusting an external parameter matrix based on the target consistency function until the target consistency function reaches the maximum value, and completing automatic calibration; The method comprises the steps of projecting points in a laser radar coordinate system of the vehicle-mounted laser radar and points in the vectorized map into an image to construct a target consistency function, projecting the points in the laser radar coordinate system into the image to obtain a first point set falling on each mask in the semantic mask set, projecting the points in the vectorized map into the image to obtain a second point set falling on each mask in the semantic mask set, and obtaining map normal vector consistency and map category consistency based on normal vector consistency, strength consistency and category consistency of the points in the first point set and the second point set together; for map normal vector consistency The formula is as follows: , Wherein, the Representing the normal vector of the lidar point P; a normal vector representing map points; representation mask The number of inner valid point pairs, P represents the points in the image mask, Q represents the corresponding points in the high-precision map; for map category consistency At each mask Under the condition, the semantic category of the projection point S of the laser radar is counted and the corresponding point set in the vector map is counted Category labels of (c) The formula is as follows: , Wherein, the Indicating an indication function, wherein the indication function is 1 when the categories are consistent, and the indication function is 0 otherwise; Is the semantic category of point P.
2. The automatic calibration method of lidar-camera external parameters according to claim 1, wherein the expression of the target consistency function is: , Wherein, the And Respectively representing normal vector consistency, intensity consistency and category consistency of points in the first point set, and commonly acquiring map normal vector consistency and map category consistency by the first point set and the second point set; and N, I, C, M, c respectively representing normal vectors, intensities and categories of points in the first point set, and commonly obtaining the normal vectors and the map categories of the map by the first point set and the second point set.
3. The automatic calibration method of external parameters of a laser radar-camera according to claim 2, wherein the adjustment formula of the external parameter matrix is: , Wherein T represents an external parameter matrix, num represents total selected num pictures, i represents an ith picture; Representing the target consistency function of the ith picture.
4. An automatic calibration device for laser radar-camera external parameters is characterized by comprising The acquisition module is used for synchronously acquiring point cloud data and image data of a road scene based on the vehicle-mounted laser radar and the camera; the generation module is used for carrying out semantic segmentation on the image data so as to generate a semantic mask set of each image; The construction module is used for extracting geometric attributes of the road scene from the vectorized map based on the point cloud data, and projecting a laser radar coordinate system of the vehicle-mounted laser radar and points in the vectorized map into an image in combination with the semantic mask set so as to construct a target consistency function; the calibration module is used for adjusting the external parameter matrix based on the target consistency function until the target consistency function reaches the maximum value, and completing automatic calibration; The construction module comprises a first projection unit, a second projection unit, an acquisition unit, a determination unit and a target consistency function, wherein the first projection unit is used for projecting points of the laser radar coordinate system into the image to obtain a first point set falling on each mask in the semantic mask set, the second projection unit is used for projecting the points in the vectorized map into the image to obtain a second point set falling on each mask in the semantic mask set, the acquisition unit is used for acquiring map normal vector consistency and map category consistency based on normal vector consistency, intensity consistency and category consistency of points in the first point set and the second point set together; for map normal vector consistency The formula is as follows: , Wherein, the Representing the normal vector of the lidar point P; a normal vector representing map points; representation mask The number of inner valid point pairs, P represents the points in the image mask, Q represents the corresponding points in the high-precision map; for map category consistency At each mask Under the condition, the semantic category of the projection point S of the laser radar is counted and the corresponding point set in the vector map is counted Category labels of (c) The formula is as follows: , Wherein, the Indicating an indication function, wherein the indication function is 1 when the categories are consistent, and the indication function is 0 otherwise; Is the semantic category of point P.
5. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the method for automatic calibration of lidar-camera parameters according to any of claims 1-3.
6. A computer-readable storage medium, on which a computer program is stored, characterized in that the program is executed by a processor for implementing the automatic calibration method of lidar-camera parameters according to any of claims 1 to 3.
7. A computer program product comprising a computer program, characterized in that the computer program is executed for implementing an automatic calibration method of lidar-camera parameters according to any of claims 1 to 3.

Description

Automatic calibration method and calibration device for laser radar-camera external parameters Technical Field The invention relates to the technical field of multi-sensor fusion, in particular to an automatic calibration method and device for external parameters of a laser radar-camera. Background In the related art, in order to realize efficient fusion of multi-sensor data, external parameter (namely relative pose relation) between a laser radar and a camera needs to be accurately calibrated. The laser radar-camera external parameter calibration method can realize accurate registration of point cloud data and image data under the same space coordinate system by estimating a rotation matrix and a translation vector between the two, thereby improving consistency and complementarity of multi-source perception information and providing a high-precision fusion foundation for downstream tasks such as target detection, environment perception, three-dimensional reconstruction and the like. However, the related technology generally depends on a large amount of labeling data or specific environmental conditions, has weak generalization capability, is difficult to adapt to complex and changeable real road scenes, cannot be effectively combined with topological structure information in a high-precision map, lacks full utilization of priori constraints on roads, lane lines and the like, and causes the problems that the calibration precision is difficult to reach the centimeter level, and is difficult to meet the severe requirements of an automatic driving system on high-precision environment sensing and fusion. Disclosure of Invention The invention provides an automatic calibration method and a calibration device for laser radar-camera external parameters, which are used for solving the technical problems that in the related technology, the calibration method for the laser radar-camera external parameters depends on a large amount of marking data or specific environmental conditions, has weak generalization capability, cannot be effectively combined with topological structure information in a high-precision map, lacks full utilization of priori constraints on roads, lane lines and the like, causes the calibration precision to be difficult to reach a centimeter level, and is difficult to meet the severe requirements of an automatic driving system on high-precision environment sensing and fusion. An embodiment of the first aspect of the invention provides an automatic calibration method of a laser radar-camera external parameter, which comprises the following steps of synchronously collecting point cloud data and image data of a road scene based on a vehicle-mounted laser radar and a camera, carrying out semantic segmentation on the image data to generate a semantic mask set of each image, extracting geometric attributes of the road scene from a vectorized map based on the point cloud data, combining the semantic mask set, projecting a laser radar coordinate system of the vehicle-mounted laser radar and points in the vectorized map into the image to construct a target consistency function, and adjusting the external parameter matrix based on the target consistency function until the target consistency function reaches the maximum value, thereby completing automatic calibration. Through the technical means, the acquired image is subjected to semantic segmentation, and the geometric attribute of the vectorized map is combined, so that a target consistency function is constructed to realize automatic external parameter calibration between the laser radar and the camera, the spatial correspondence between the image semantic and the map geometric can be fully utilized, the accuracy and the robustness of feature matching are improved, the automation degree and the environment adaptability of the multi-mode sensor calibration are effectively improved, the dependence on manual intervention and priori samples is reduced, and the perception fusion capability of the system under a complex traffic scene is enhanced. Optionally, in one embodiment of the present invention, the projecting the points in the lidar coordinate system and the vectorized map of the vehicle-mounted lidar into an image includes projecting the points of the lidar coordinate system into the image to obtain a first set of points falling on each mask in the set of semantic masks, and projecting the points in the vectorized map into the image to obtain a second set of points falling on each mask in the set of semantic masks. By the technical means, the points in the laser radar coordinate system and the points in the vectorized map are respectively projected to the image to obtain the point set corresponding to the image semantic mask, so that semantic alignment and spatial association of different data sources under the unified image coordinate system can be realized, the multi-mode fusion capability among the image, the point cloud and the map is enhanced, the a