CN-122023655-A - Three-dimensional reconstruction method, system and terminal equipment for underwater binocular system

CN122023655ACN 122023655 ACN122023655 ACN 122023655ACN-122023655-A

Abstract

The invention discloses a three-dimensional reconstruction method of an underwater binocular system, and relates to the technical field of underwater optical three-dimensional measurement. The method comprises the steps of receiving camera internal parameters and distortion coefficients calibrated by a left camera and a right camera in an underwater binocular system in the air and relative external parameters between the two cameras, receiving plane equation parameters of an external plane of a cabin lens of the underwater binocular system under a left camera coordinate system and synchronous left and right image pairs of an object to be tested acquired by the underwater binocular system in an underwater environment, wherein the plane equation parameters comprise a normal vector n and an origin to plane distance d, preprocessing the acquired left and right image pairs, calculating a parallax image or a characteristic point corresponding relation by a stereo matching algorithm, calibrating and correcting front and rear point cloud initial errors by a virtual image precalibration method, and adjusting camera focusing parameters. According to the invention, by accurately calibrating cabin mirror interface parameters and combining strict optical path tracking and Snell's law, a physically accurate multi-refraction correction model is constructed, so that high-precision correction is performed on the basis of traditional binocular reconstruction, and finally, the accurate three-dimensional coordinates of a real underwater object are obtained.

Inventors

WU TAO
HOU SHITONG
LI ZHENG
WU ZHISHEN
HE XIAOYUAN

Assignees

东南大学

Dates

Publication Date: 20260512
Application Date: 20260130

Claims (10)

1. The three-dimensional reconstruction method of the underwater binocular system is characterized by comprising the following steps of: S1, receiving camera internal parameters and distortion coefficients of left and right cameras calibrated in the air in an underwater binocular system and relative external parameters between the two cameras, wherein the relative external parameters comprise a rotation matrix R and a translation vector T; s2, receiving plane equation parameters of an external plane of a cabin lens of the underwater binocular system under a left camera coordinate system and synchronous left and right image pairs of an object to be detected acquired by the underwater binocular system in an underwater environment, wherein the plane equation parameters comprise a normal vector n and an origin to plane distance d; S3, preprocessing the acquired left and right image pairs, calculating the corresponding relation of parallax images or characteristic points by adopting a stereo matching algorithm, and reconstructing to obtain an initial three-dimensional point cloud P initial of the measured object under a left camera coordinate system based on the traditional binocular triangulation principle; S4, for any point P i in the initial three-dimensional point cloud P initial , connecting lines of the point P i and optical centers of the left camera and the right camera are respectively constructed, and a space straight line L L under a left camera coordinate system and a space straight line L R under a right camera coordinate system are formed; S5, respectively carrying out reverse light path tracking on the left camera and the right camera based on the image point coordinates and the camera parameters calibrated in the step S1, namely calculating the incident direction of light rays from the camera optical center on the inner surface of the cabin mirror, sequentially calculating the first refraction direction of the light rays on the air-glass interface on the inner surface of the cabin mirror and the second refraction direction of the light rays on the glass-water interface on the outer surface of the cabin mirror according to the Snell' S law and the plane equation parameters calibrated in the step S2, finally obtaining the underwater light ray direction v water , and constructing a deterministic calculation relation from the image point coordinates p L to the underwater light ray direction v water , namely a point cloud refraction correction model; S6, converting an underwater light equation of the right camera into a left camera coordinate system by utilizing the relative external parameters calibrated in the step S1, so as to obtain two underwater light equations corresponding to the same object point under the left camera coordinate system, specifically a left camera light equation and a right camera light equation, wherein the left camera light equation is based on O L and v water_L , and the right camera light equation is based on O R and v water_R ; S7, calculating the optimal intersection point coordinates P corrected (X c , Y c , Z c between the two light rays of the left camera and the right camera by adopting a least square method, traversing all matching point pairs as three-dimensional point coordinates corrected by the multi-refraction model, and repeating the steps S4 to S8 to obtain a final corrected three-dimensional point cloud { P corrected }; And S8, based on the established point cloud refraction correction model, carrying out an underwater virtual image pre-calibration test of the measured object, measuring initial errors of the point cloud before and after correction, adjusting focusing parameters of the cameras, and carrying out calibration of internal parameters of the cameras, distortion coefficients, relative external parameters between the two cameras and an external plane equation of a cabin lens of an underwater binocular system.
2. The method of three-dimensional reconstruction of an underwater binocular system of claim 1, wherein the underwater binocular vision system comprises a left camera, a right camera and a front plane mirror which are sealed in a waterproof shell.
3. The method for reconstructing the underwater binocular system in three dimensions according to claim 2, wherein the calibration of the plane equation parameters is realized by adopting a subtended auxiliary binocular measuring system, and the method comprises the following steps: constructing an auxiliary binocular camera system with known internal parameters and relative external parameters on the right opposite side of the underwater binocular system; using a three-dimensional calibration object with a coding point attached to the surface, and measuring the accurate three-dimensional coordinates of the coding point under a global coordinate system by using photography; the method comprises the steps that a calibration object is placed in a common view field of two sets of systems, and three-dimensional coordinates of a coding point under respective coordinate systems are rebuilt through an underwater binocular system and an auxiliary binocular system respectively; based on the coded points with the same name, calculating the coordinate conversion relation from the auxiliary binocular system coordinate system to the underwater binocular system coordinate system; And carrying out three-dimensional reconstruction on the characteristic points adhered to the outer surface of the cabin mirror through an auxiliary binocular system, and converting the coordinates of the characteristic points into an underwater binocular system coordinate system by utilizing a coordinate conversion relation to carry out plane fitting, so as to obtain plane equation parameters.
4. The three-dimensional reconstruction method of an underwater binocular system according to claim 3, wherein in step S3, a disparity map or a feature point correspondence is calculated by adopting a stereo matching algorithm, and based on the traditional binocular triangulation principle, an initial three-dimensional point cloud P initial of a measured object under a left camera coordinate system is obtained by reconstruction, specifically as follows: For a matching point pair, its three-dimensional coordinates are obtained by solving the following equation: (1) (2) where S L 、S R is the scale factor, the calculation does not take into account the refraction of the light at the capsule, so the result is an initial coordinate with errors.
5. The three-dimensional reconstruction method of an underwater binocular system according to claim 4, wherein the construction of the point cloud refraction correction model is specifically as follows: Let the plane equation of the capsule be n·p=d, P be any point on the plane, the incident ray with the direction v air from the optical center O intersects the plane at point P glass_in , and the direction v glass of the ray refracted into the glass satisfies the following snell's law: (3) Wherein n air is the refractive index of air, x represents the cross product; similarly, at the glass-water interface (assuming that the inner and outer surfaces are parallel, the normal vector is n), the light ray direction v water refracted into water from the direction v glass satisfies: (4) V air is calculated according to the pixel coordinates (u, v) and the internal reference K, and then the final underwater ray direction v water is calculated in a recurrence mode.
6. The three-dimensional reconstruction method of an underwater binocular system according to claim 5, wherein in the step S7, the optimal intersection point of two space light rays is calculated by adopting a least square method, specifically: Let the left camera ray equation be p=o L + t L V L the right camera ray equation converted to the left coordinate system is p=o R + t R ՛ V R ՛, where v L and v R ՛ are normalized direction vectors, and t L and t R are scalar parameters; Parameters t L and t R are found such that two points P L = O L + t L V L and P R = O R ՛+ t R V R ՛, converting the problem into solving a linear equation set, and optimally solving the problem into a midpoint corresponding to the time when the square sum of the distances is minimum or a point obtained by solving a normal equation, wherein the point coordinate is the corrected three-dimensional point P corrected .
7. The three-dimensional reconstruction method of an underwater binocular system according to claim 6, wherein after the initial three-dimensional point cloud P initial of the object under test in the left camera coordinate system is obtained by reconstruction, outlier rejection or filtering operation is further required for the initial point cloud P initial .
8. An underwater binocular system three-dimensional reconstruction system for implementing an underwater binocular system three-dimensional reconstruction method as set forth in any one of claims 1 to 7, comprising: The underwater binocular vision acquisition assembly comprises an underwater binocular camera system for completing cabin mirror parameter calibration, a waterproof shell and a lighting device; The underwater carrying and positioning assembly is specifically an ROV, an AUV or a fixed underwater platform and is used for carrying the acquisition module and controlling the movement or the pose of the acquisition module relative to the object to be detected; The data processing and reconstructing component specifically comprises: The parameter calibration module is used for receiving camera internal parameters and distortion coefficients calibrated by the left camera and the right camera in the underwater binocular system in the air and relative external parameters between the two cameras, wherein the relative external parameters comprise a rotation matrix R and a translation vector T; The image acquisition module is used for receiving plane equation parameters of an external plane of a cabin lens of the underwater binocular system under a left camera coordinate system and synchronous left and right image pairs of an object to be detected acquired by the underwater binocular system in an underwater environment, wherein the plane equation parameters comprise a normal vector n and an origin to plane distance d; The initial three-dimensional point cloud construction module is used for preprocessing the acquired left and right image pairs, calculating the parallax image or the characteristic point corresponding relation by adopting a stereo matching algorithm, and reconstructing to obtain an initial three-dimensional point cloud P initial of the measured object under the left camera coordinate system based on the traditional binocular triangulation principle; The space straight line construction module is used for respectively constructing connection lines of the initial three-dimensional point cloud P initial and optical centers of the left camera and the right camera for any point P i in the initial three-dimensional point cloud P initial to form a space straight line L L under a left camera coordinate system and a space straight line L R under a right camera coordinate system; The point cloud refraction correction model construction module is used for respectively carrying out reverse light path tracking on the left camera and the right camera based on image point coordinates and calibrated camera parameters, namely calculating the incident direction of light rays emitted from a camera optical center on the inner surface of the cabin mirror, sequentially calculating the first refraction direction of the light rays on the air-glass interface on the inner surface of the cabin mirror and the second refraction direction of the light rays on the glass-water interface on the outer surface of the cabin mirror according to the Snell's law and calibrated plane equation parameters, finally obtaining an underwater light ray direction v water , and constructing a deterministic calculation relation from the image point coordinates p L to the underwater light ray direction v water , namely a point cloud refraction correction model; The underwater light equation conversion module is used for converting the underwater light equation of the right camera into a left camera coordinate system by using a calibrated relative external parameter, so that two underwater light equations corresponding to the same object point are obtained in the left camera coordinate system, namely a left camera light equation and a right camera light equation, wherein the left camera light equation is based on O L and v water_L , and the right camera light equation is based on O R and v water_R ; The solving and outputting module is used for calculating the optimal intersection point coordinates P corrected (X c , Y c , Z c between the two light rays of the left camera and the right camera by adopting a least square method, and traversing all matching point pairs to obtain a final corrected three-dimensional point cloud { P corrected }; The error correction module is used for carrying out an underwater measured object virtual image precalibration test based on the established point cloud refraction correction model, measuring the initial error of the point cloud before and after correction, adjusting focusing parameters of the cameras, and carrying out calibration on internal parameters of the cameras, distortion coefficients, relative external parameters between the two cameras and an external plane equation of a cabin lens of an underwater binocular system.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, characterized in that the processor, when loading and executing the computer program, employs the underwater binocular system three-dimensional reconstruction method according to any of claims 1 to 7.
10. A storage medium containing computer executable instructions which, when executed by a computer processor, are for performing the underwater binocular system three-dimensional reconstruction method of any one of claims 1 to 7.

Description

Three-dimensional reconstruction method, system and terminal equipment for underwater binocular system Technical Field The invention relates to the technical field of underwater optical three-dimensional measurement, in particular to an underwater binocular system three-dimensional reconstruction method, an underwater binocular system three-dimensional reconstruction system and terminal equipment. Background The underwater binocular stereoscopic vision is an important non-contact measurement means for acquiring three-dimensional morphological information of an underwater object, and is widely applied to the fields of environment sensing and operation guidance of underwater robots such as marine resource exploration, underwater engineering structure detection, underwater archaeology, aquatic organism research and ROV/AUV. The basic principle is that human binocular parallax is simulated through two sets of imaging light paths, and three-dimensional space information is recovered from a two-dimensional image by using a triangulation principle. However, underwater binocular systems typically require the camera to be placed in a watertight pressure cabin, and observed through a flat glass or acrylic window (cabin mirror). When light enters the camera from the aqueous medium through the capsule, two refractions occur at the water-glass-air interface, which seriously violates the assumptions of the traditional pinhole camera model and the principle of binocular triangulation. The underwater three-dimensional reconstruction is directly carried out by using calibrated parameters in the air, systematic errors which change along with the position of an object point can be generated, and the reconstruction point cloud is distorted, the scale distortion and the precision are reduced. The prior art attempts to solve the problem, mainly divided into two types, namely, refractive model correction based on accurate physical parameters and equivalent model approximation through underwater specific calibration. The former requires precise knowledge of the geometric parameters (thickness, refractive index) and spatial attitude of the capsule, which is difficult to guarantee in engineering assembly, and the model is complex and computationally intensive. The latter (such as a virtual camera model) simplifies the calculation, but the calibration process is complicated, the precision is limited by the calibration range and the precision of the calibration plate, and the model generalization capability is limited. Therefore, the invention provides a three-dimensional reconstruction method, a three-dimensional reconstruction system and terminal equipment for an underwater binocular system. Disclosure of Invention The invention aims to provide a three-dimensional reconstruction method, a three-dimensional reconstruction system and a three-dimensional reconstruction terminal device for an underwater binocular system, wherein a physically accurate multi-refraction correction model is constructed by precisely calibrating cabin mirror interface parameters and combining strict optical path tracking and Snell's law, so that high-precision correction is performed on the basis of traditional binocular reconstruction, and finally, the precise three-dimensional coordinates of a real underwater object are obtained. According to the first aspect of the invention, in order to achieve the above object, the invention provides a three-dimensional reconstruction method of an underwater binocular system, comprising the following steps: S1, receiving camera internal parameters and distortion coefficients of left and right cameras calibrated in the air in an underwater binocular system and relative external parameters between the two cameras, wherein the relative external parameters comprise a rotation matrix R and a translation vector T; s2, receiving plane equation parameters of an external plane of a cabin lens of the underwater binocular system under a left camera coordinate system and synchronous left and right image pairs of an object to be detected acquired by the underwater binocular system in an underwater environment, wherein the plane equation parameters comprise a normal vector n and an origin to plane distance d; S3, preprocessing the acquired left and right image pairs, calculating the corresponding relation of parallax images or characteristic points by adopting a stereo matching algorithm, and reconstructing to obtain an initial three-dimensional point cloud P initial of the measured object under a left camera coordinate system based on the traditional binocular triangulation principle; S4, for any point P i in the initial three-dimensional point cloud P initial, connecting lines of the point P i and optical centers of the left camera and the right camera are respectively constructed, and a space straight line L L under a left camera coordinate system and a space straight line L R under a right camera coordinate system are formed; S5,