CN-122023537-A - Robust multi-camera system calibration method and system suitable for low-quality images

CN122023537ACN 122023537 ACN122023537 ACN 122023537ACN-122023537-A

Abstract

The invention belongs to the technical field of computer vision and photogrammetry, and particularly relates to a method and a system for calibrating a robust multi-camera system which is suitable for low-quality images, wherein after initial external parameters of all cameras are determined based on angular point detection data, non-reference cameras with larger baseline lengths between front K non-reference cameras and reference cameras are selected, K optimal triangulation camera pairs are respectively formed by the non-reference cameras and the reference cameras, and the angular point detection data are subjected to triangulation to generate K candidate three-dimensional point cloud models; and fusing the K candidate three-dimensional point cloud models to obtain a consensus three-dimensional point cloud model, solving the reconstruction external parameters of the corresponding cameras based on the consensus three-dimensional point cloud model, taking the reconstruction external parameters of all cameras and the consensus three-dimensional point cloud model as the input of the global BA optimizer, and performing iterative optimization to complete external parameter calibration. The invention provides robust and high-quality input for the global BA optimizer through reconstructing the external parameters and the consensus three-dimensional point cloud model, and finally realizes the global consistent and high-precision external parameter calibration.

Inventors

ZHANG SHU
CHEN JIAPENG
HE HAIXIANG
YANG SONGMAO
Li Daixiang

Assignees

华中农业大学

Dates

Publication Date: 20260512
Application Date: 20251210

Claims (10)

1. A robust multi-camera system calibration method adapting to low-quality images is characterized in that: The calibration method of the multi-camera system comprises the following steps: s1, acquiring a plurality of groups of calibration images by using a multi-camera system to form a candidate set; s2, detecting corner points of each image in the candidate set; S3, designating one camera as a reference camera in the multi-camera system, using other cameras as non-reference cameras, obtaining angular point data by using the S2, and calculating initial external parameters of all cameras through three-dimensional calibration; S4, calculating the base line length between all non-reference cameras and the reference camera, selecting the non-reference cameras with larger base line length between the first K non-reference cameras and the reference camera as optimal far-end cameras, and respectively forming K optimal triangulation camera pairs by the K optimal far-end cameras and the reference camera; S5, performing triangulation on the angular point data by using K optimal triangulation cameras respectively to generate K candidate three-dimensional point cloud models, and fusing the K candidate three-dimensional point cloud models to obtain a consensus three-dimensional point cloud model; s6, taking the reconstructed external parameters of all cameras and the consensus three-dimensional point cloud model as input of a global BA optimizer, and obtaining final external parameters of all cameras by iterative optimization of the global BA optimizer to finish external parameter calibration of the multi-camera system.
2. A method for calibrating a robust multi-camera system adapted to low quality images according to claim 1, wherein: In the step S5, K candidate three-dimensional point cloud models are fused by adopting a least square method based on iterative re-weighting; The least square method based on iterative re-weighting comprises the steps of firstly setting corresponding initial quality weights for each candidate three-dimensional point cloud model, calculating weighted average models of K candidate three-dimensional point cloud models based on the initial quality weights, then constructing a least square optimization problem, which is the distance from a minimum consensus three-dimensional point cloud model to the weighted average model, by using the weighted average model as the current optimal estimation of the least square optimization problem, iteratively updating the quality weights, re-calculating the weighted average model based on the updated quality weights until the iteration ending condition is met, and outputting the weighted average model obtained in the last iteration as the consensus three-dimensional point cloud model.
3. A method for calibrating a robust multi-camera system adapted to low quality images according to claim 2, wherein: Setting an initial quality weight of each candidate three-dimensional point cloud model according to the following formula: ; ; ; In the above-mentioned method, the step of, 、、 Respectively obtaining initial quality weight, baseline weight and reprojection error weight of the kth candidate three-dimensional point cloud model; the base line length corresponding to the kth candidate three-dimensional point cloud model is set; Representing the average re-projection error of the two corresponding cameras for back-projecting the kth candidate three-dimensional point cloud model; A small constant for zero control.
4. A method of calibrating a robust multi-camera system adapted to low quality images according to any of claims 1-3, characterized in that: In the S6, the step of iterative optimization of the global BA optimizer comprises the steps of constructing a total error function of the global BA optimizer, and performing iterative optimization on the total error function by using a nonlinear least square method until the total error function converges to obtain final external parameters of all cameras, wherein the total error function comprises the sum of errors of projection of consensus three-dimensional point clouds of all cameras under all views onto respective two-dimensional images.
5. A method of calibrating a robust multi-camera system adapted to low quality images according to any of claims 1-3, characterized in that: The step S2 comprises the following steps: s21, performing corner detection on each image in the candidate set by adopting a corner detection algorithm according to a preset maximum detection size, triggering size reduction if the corner detection fails, and performing corner detection again by using the detection size after the size reduction until the corner detection is successful or the detection size after the size reduction is smaller than a preset minimum effective threshold; S22, obtaining detection sizes of all images, determining a common area which can be commonly observed by all cameras based on a geometric intersection of the detection sizes of all images under a calibration plate coordinate system, and only preserving corner data belonging to the common area.
6. A robust multi-camera system calibration system adapting to low-quality images is characterized in that: The multi-camera system calibration system comprises: the image acquisition module is used for acquiring a plurality of groups of calibration images from the calibration plate by utilizing the multi-camera system to form a candidate set; The corner detection module is used for detecting the corner of each image in the candidate set; The initial external parameter calculation module is used for designating one camera as a reference camera in the multi-camera system, designating other cameras as non-reference cameras, and calculating initial external parameters of all cameras through three-dimensional calibration by utilizing angular point data; The optimal far-end camera selection module is used for calculating the base line length between all the non-reference cameras and the reference camera, selecting the non-reference cameras with larger base line length between the first K non-reference cameras and the reference camera as the optimal far-end cameras, and respectively forming K optimal triangulation camera pairs by the K optimal far-end cameras and the reference camera; The reconstruction external parameter calculation module is used for carrying out triangulation on the angular point data by using K optimal triangulation cameras to generate K candidate three-dimensional point cloud models, and fusing the K candidate three-dimensional point cloud models to obtain a consensus three-dimensional point cloud model; And the camera calibration module is used for taking the reconstructed external parameters of all cameras and the consensus three-dimensional point cloud model as the input of the global BA optimizer, and obtaining the final external parameters of all cameras by iterative optimization of the global BA optimizer so as to finish the external parameter calibration of the multi-camera system.
7. A robust multi-camera system calibration system for adapting to low quality images according to claim 6, characterized in that: The reconstruction extrinsic parameter calculation module is used for fusing K candidate three-dimensional point cloud models by adopting a least square method based on iterative re-weighting, wherein the least square method based on iterative re-weighting comprises the steps of firstly setting corresponding initial mass weights for each candidate three-dimensional point cloud model, calculating weighted average models of the K candidate three-dimensional point cloud models based on the initial mass weights, then constructing a least square optimization problem and using the weighted average model as the current optimal estimation of the least square optimization problem, wherein the least square optimization problem is the distance from the minimum consensus three-dimensional point cloud model to the weighted average model, iteratively updating the mass weights and re-calculating the weighted average model based on the updated mass weights until the iteration ending condition is met, and outputting the weighted average model obtained in the last iteration as the consensus three-dimensional point cloud model.
8. A robust multi-camera system calibration system for adapting to low quality images according to claim 7, wherein: The reconstruction extrinsic parameters calculation module is used for setting the initial quality weight of each candidate three-dimensional point cloud model according to the following formula: ; ; ; In the above-mentioned method, the step of, 、、 Respectively obtaining initial quality weight, baseline weight and reprojection error weight of the kth candidate three-dimensional point cloud model; the base line length corresponding to the kth candidate three-dimensional point cloud model is set; Representing the average re-projection error of the two corresponding cameras for back-projecting the kth candidate three-dimensional point cloud model; A small constant for zero control.
9. A robust multi-camera system calibration system for adapting to low quality images according to any one of claims 6 to 8, characterized in that: The camera calibration module is used for carrying out iterative optimization on the global BA optimizer, wherein the global BA optimizer is constructed with a total error function, the total error function is subjected to iterative optimization by using a nonlinear least square method until the total error function converges to obtain final external parameters of all cameras, and the total error function comprises the sum of errors of projection of consensus three-dimensional point clouds of all cameras under all views onto respective two-dimensional images.
10. A robust multi-camera system calibration system for adapting to low quality images according to any one of claims 6 to 8, characterized in that: Preferably, the angular point detection module comprises a dimension reduction module and a data processing module; The dimension reduction module is used for carrying out angle point detection on each image in the candidate set by adopting an angle point detection algorithm according to a preset maximum detection dimension, if the angle point detection fails, dimension reduction is triggered, and the dimension reduction is carried out again by using the dimension reduced detection dimension until the angle point detection is successful or the dimension reduced detection dimension is smaller than a preset minimum effective threshold; the data processing module is used for obtaining the detection sizes of all the images, determining a common area which can be commonly observed by all cameras based on the geometric intersection of the detection sizes of all the images under the coordinate system of the calibration plate, and only preserving the corner data belonging to the common area.

Description

Robust multi-camera system calibration method and system suitable for low-quality images Technical Field The invention belongs to the technical field of computer vision and photogrammetry, and particularly relates to a method, a system, equipment and a medium for calibrating a robust multi-camera system suitable for low-quality images. Background In a multi-camera three-dimensional vision system, joint calibration is generally required to obtain accurate positions and attitudes (i.e., external parameters) of all cameras in the same world coordinate system. The existing joint calibration flow is that a series of images are shot in a common view field of a plurality of cameras by using a calibration object (such as a checkerboard), two-dimensional pixel coordinates of characteristic points (such as angular points) of the calibration object in each image are detected, one reference camera is selected, initial estimation of external parameters (including a rotation matrix and translation vectors) of other cameras relative to the reference camera is completed through three-dimensional calibration (stereoCalibrate) between two cameras, a pair of cameras (such as a reference camera 1 and a non-reference camera 2) is selected for establishing an initial three-dimensional point cloud, three-dimensional corresponding points of the cameras are reconstructed into three-dimensional space points, the initial three-dimensional points of the initial three-dimensional points and the initial external parameters of all cameras are taken as initial values, global optimization (global optimization is realized through binding adjustment generally), and errors of re-projection of all three-dimensional points in all cameras to the two-dimensional images are minimized, so that final accurate calibration parameters are obtained. However, the above detection method has the following problems: 1. Corner detection is performed in a complete calibration plate pattern (e.g. a complete 8x11 checkerboard) of predefined dimensions. In actual data acquisition, a large number of calibration images cannot be completely shot due to shooting angles, distances or object shielding and other reasons. Although these images may contain more than 90% of valid corner information, they are determined to be invalid and discarded directly because the detection requirements of the full mode are not met. The one-ticket overrule type rigid mechanism causes huge data waste and seriously reduces the utilization rate of the calibration image. In a scenario where acquisition conditions are limited, the entire calibration process may even fail due to a sufficient number of "perfect" images being jagged. Therefore, the prior art has the problems that the requirements on the image quality are harsh, a large amount of calibration images are wasted, and the calibration images are difficult to apply to calibration scenes under non-ideal imaging conditions (such as uneven illumination, smaller area of a calibration plate, disordered background scenes and slight defocus blur of the calibration images). 2. To obtain high precision camera parameters, the multi-camera calibration will eventually typically perform a global binding optimization step, the success of which is highly dependent on the quality of the initial external parameters (i.e. initial camera pose and three-dimensional point coordinates). The current detection method generally adopts a static initialization strategy, namely, an initial pose of all cameras relative to a reference camera is calculated, a pair of cameras (for example, a reference camera 1 and a non-reference camera 2 are used for constructing a camera pair, or a longest baseline pair is selected for constructing a camera pair based on a long baseline-high precision principle) is fixedly selected for triangulation, an initial three-dimensional point cloud is generated, the selected camera pair has an estimation error on the initial relative pose, the pose error can be directly transmitted and amplified to all the initial three-dimensional point clouds by using the pose with the error for triangulation, and the initial three-dimensional point cloud with the topological structure error is used as a starting point of a BA algorithm, so that the optimization process is very easy to be incapable of converging and converging to an erroneous local optimal solution or is very long in time consumption. Disclosure of Invention The invention aims to solve the problem of low external parameter calibration precision caused by estimation errors of pose of a single camera in the prior art, and provides a robust multi-camera system calibration method, a system, equipment and a medium for realizing high-precision external parameter calibration by fusing three-dimensional point clouds of multiple camera pairs to provide robust and high-quality three-dimensional point cloud input for global BA optimization. In order to achieve the above object, the technical sche