CN-122023461-A - High-precision three-dimensional reconstruction method of dynamic scene based on inter-frame geometric constraint and motion tracking

CN122023461ACN 122023461 ACN122023461 ACN 122023461ACN-122023461-A

Abstract

The invention discloses a dynamic scene high-precision three-dimensional reconstruction method based on interframe geometric constraint and motion tracking, which comprises the steps of system calibration and data acquisition; the method comprises the steps of wrapping phase calculation, interframe geometric constraint phase expansion based on motion tracking, phase unwrapping correction based on confidence driving, three-dimensional coordinate reconstruction, information updating and iteration. The method transmits reliable three-dimensional information of a historical frame to a current frame through motion tracking, provides dynamic self-adaptive reference for geometric constraint unwrapping, fundamentally solves the problem of pixel misalignment caused by motion, is suitable for dynamic scene three-dimensional measurement of continuous motion, and combines inter-frame geometric constraint and motion tracking to improve the accuracy of phase unwrapping. The introduced confidence driving correction mechanism can effectively identify and repair abnormal points generated by motion estimation errors, shielding, depth jump and the like, and the anti-interference capability and the overall reconstruction accuracy of the algorithm under a complex scene are remarkably improved.

Inventors

BAI LIANFA
LIU XIN
YU HAOTIAN
HAN JING
ZHENG DONGLIANG
LIU XINYI

Assignees

南京理工大学

Dates

Publication Date: 20260512
Application Date: 20260403

Claims (8)

1. A dynamic scene high-precision three-dimensional reconstruction method based on inter-frame geometric constraint and motion tracking is characterized by comprising the following steps: The method comprises the steps of firstly, constructing an FPP system consisting of a camera and a projector, completing system calibration, and obtaining internal and external parameters of the camera and the projector; step two, calculating a wrapping phase diagram of the current frame by using a phase shift method for the acquired single group of phase shift stripe patterns; step three, motion tracking is carried out based on the motion continuity between adjacent frames, and reliable geometric constraint reference is provided for the current frame, so that absolute phases are solved; Aiming at the unreliable absolute phase result in the third step, performing phase unwrapping correction based on confidence evaluation to obtain an optimized absolute phase diagram of the current frame; calculating three-dimensional space coordinates of each point on the surface of the object by utilizing the absolute phase diagram of the current frame and combining system calibration parameters and by using a triangulation principle, and completing three-dimensional reconstruction of the current frame; And step six, storing and updating the three-dimensional point cloud obtained by reconstructing the current frame and the corresponding absolute phase information as the reference information of the previous frame for processing the next frame of image, thereby realizing real-time three-dimensional reconstruction in a continuous dynamic scene.
2. The method for high-precision three-dimensional reconstruction of dynamic scene based on inter-frame geometric constraint and motion tracking according to claim 1, wherein the third step of motion tracking based on motion continuity between adjacent frames provides reliable geometric constraint reference for the current frame so as to solve absolute phase, specifically comprises: 1) Estimating the rigid body motion of the object from the previous frame to the current frame by utilizing the reconstructed three-dimensional point cloud information of the previous frame and combining the wrapping phase of the current frame and the geometric constraint relation of the system, and obtaining a motion matrix from the previous frame to the current frame; 2) Transforming the absolute phase of the previous frame to the view angle of the current frame by using the estimated motion matrix, and generating a predicted absolute phase reference value for each pixel of the current frame; 3) Comparing the wrapped phase of each pixel of the current frame with the absolute phase reference value predicted in the step 2), wherein the absolute phase and the wrapped phase are different according to the principle of geometric constraint By finding the difference between them at integer times Integer multiple in interval, determining the fringe order of the pixel Further, the absolute phase is calculated Wherein To wrap the phase.
3. The method for high-precision three-dimensional reconstruction of dynamic scene based on inter-frame geometric constraint and motion tracking according to claim 2, wherein in the third step, a nearest neighbor iterative optimization method is adopted to optimize and solve a motion matrix by minimizing the distance between the corresponding points of the preliminary three-dimensional point set generated by wrapping phase and geometric constraint of the current frame and the three-dimensional point set of the previous frame after motion transformation.
4. The method according to claim 2, wherein in the third step, when the absolute phase is solved by using the geometric constraint, a threshold value of the allowable depth variation range based on the system parameter is set, and when the three-dimensional depth difference calculated by the predicted absolute phase reference value and a certain candidate absolute phase solution is within the threshold value, the candidate solution is considered to be reasonable.
5. The method for high-precision three-dimensional reconstruction of a dynamic scene based on inter-frame geometric constraint and motion tracking according to claim 1, wherein the fourth step is to perform phase unwrapping correction based on confidence evaluation for the unreliable absolute phase result in the third step to obtain an optimized absolute phase map of the current frame, and specifically comprises the following steps: 1) According to the degree of coincidence between the predicted absolute phase reference value and the final selected solution, the consistency of the neighborhood phases and the continuity information of the inter-frame phases when the absolute phases are solved in the third step, a confidence score is given to the unwrapped result of each pixel; 2) The pixels with the confidence coefficient lower than the set threshold value are judged to be unreliable points or potential error points, isolation is carried out, and errors of the pixels are prevented from affecting other reliable areas; 3) And for the isolated low-confidence region, carrying out phase correction and filling by using the phase information of the high-confidence point in the space neighborhood of the isolated low-confidence region or using the information of the high-confidence region of the previous frame in time sequence to obtain an absolute phase diagram.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1-5 when executing the program.
7. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any of claims 1-5.
8. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any of claims 1-5.

Description

High-precision three-dimensional reconstruction method of dynamic scene based on inter-frame geometric constraint and motion tracking Technical Field The invention belongs to the field of optical three-dimensional measurement and computer vision, and particularly relates to a dynamic scene high-precision three-dimensional reconstruction method based on inter-frame geometric constraint and motion tracking. Background The optical three-dimensional measurement technology is widely applied to the fields of industrial detection, intelligent manufacturing, reverse engineering, medical diagnosis, cultural heritage protection and the like. The structured light three-dimensional measurement technology based on stripe coding, especially stripe projection profile (Fringe Projection Profilometry, FPP), is a current research and application hotspot because of the advantages of non-contact, high precision, high efficiency, relatively simple system structure and the like. In FPP, in order to acquire three-dimensional information of an object, it is necessary to project an encoded fringe pattern onto the surface of the object and capture, by a camera, deformed fringes modulated by the topography of the object. And (3) through resolving the phase information of the deformation stripes, establishing a corresponding relation between the camera and the projector pixels, and further recovering the three-dimensional coordinates of the object based on the triangulation principle. However, the phase directly calculated by the phase shift method or the like is wrapped aroundThe wrapping phase in the interval has periodicity and cannot be directly used for unique matching. Thus, a phase unwrapping (Phase Unwrapping) is required to obtain an absolute, continuous phase value, i.e., absolute phase. The conventional phase unwrapping method is mainly divided into temporal phase unwrapping and spatial phase unwrapping. The time phase expansion (such as a multi-frequency heterodyne method and a gray code method) needs to project a plurality of coding patterns, and a high-precision result can be obtained in a static scene. However, in a dynamic scene, a plurality of patterns projected continuously due to the movement of an object cannot be aligned at the pixel level, which may introduce serious movement errors, resulting in unwrapping failure. The spatial phase unwrapping only requires a single phase map, but relies on the assumption of smooth surfaces, is less robust in the presence of depth jumps, isolated objects or noise disturbances, and is prone to erroneous transfer. To address the challenges of dynamic scenes, prior studies have attempted by reducing the number of projected patterns (e.g., complex fringes, color coding), introducing additional hardware (e.g., a multi-view system), or utilizing a priori information (e.g., CAD models), etc. However, these methods often have problems of high hardware cost, complex system, limited applicable scene, weak anti-interference capability, and the like. In particular, for a standard single camera-single projector FPP system, how to realize reliable and high-precision phase expansion and three-dimensional reconstruction in a motion scene without projecting additional unwrapping patterns and increasing hardware complexity is still a technical problem to be solved. Disclosure of Invention The invention aims to overcome the defects of the prior art, and provides a dynamic scene high-precision three-dimensional reconstruction method based on inter-frame geometric constraint and motion tracking, which is used for optimizing and correcting an unwrapping result by combining a confidence driving mechanism, so that the rapid and accurate three-dimensional reconstruction of a moving object is realized under the condition that additional projection unwrapping patterns and hardware support are not needed. The technical scheme for realizing the purpose of the invention is that the high-precision three-dimensional reconstruction method of the dynamic scene based on the inter-frame geometric constraint and the motion tracking comprises the following steps: the method comprises the steps of firstly, constructing an FPP system consisting of a camera and a projector, completing system calibration, and obtaining internal and external parameters of the camera and the projector; step two, calculating a wrapping phase diagram of the current frame by using a phase shift method for the acquired single group of phase shift fringe images; step three, motion tracking is carried out based on the motion continuity between adjacent frames, and reliable geometric constraint reference is provided for the current frame, so that absolute phases are solved; Aiming at the unreliable absolute phase result in the third step, performing phase unwrapping correction based on confidence evaluation to obtain an optimized absolute phase diagram of the current frame; calculating three-dimensional space coordinates of each point on the sur