CN-116188688-B - Three-dimensional reconstruction method, three-dimensional reconstruction device, and computer-readable storage medium

CN116188688BCN 116188688 BCN116188688 BCN 116188688BCN-116188688-B

Abstract

The application provides a three-dimensional reconstruction method, a three-dimensional reconstruction device and a computer readable storage medium. The three-dimensional reconstruction method comprises the steps of obtaining a modulation fringe image, obtaining a wrapping phase based on the modulation fringe image, obtaining an infrared image based on the modulation fringe image, obtaining a first three-dimensional point cloud by utilizing the infrared image and a visible light image, obtaining a first absolute phase by utilizing the first three-dimensional point cloud and the wrapping phase, and reconstructing a second three-dimensional point cloud of an object according to the modulation fringe image and the first absolute phase. According to the three-dimensional reconstruction device, the infrared projector and the infrared camera are used for collecting projection fringe patterns, so that the noninductive three-dimensional reconstruction can be realized, the two vision subsystems are used for three-dimensional reconstruction, the information obtained by the two vision subsystems is fully utilized, the three-dimensional reconstruction result is fused, and the speed and the precision of the three-dimensional reconstruction can be greatly improved.

Inventors

ZHANG ZHAO
WANG PENGFEI

Assignees

杭州华橙软件技术有限公司

Dates

Publication Date: 20260508
Application Date: 20230118

Claims (9)

1. A three-dimensional reconstruction method, characterized in that the three-dimensional reconstruction method comprises: Acquiring a modulation fringe image, and acquiring a wrapping phase based on the modulation fringe image; Acquiring an infrared image based on the modulated stripe image; Acquiring a first three-dimensional point cloud by utilizing the infrared image and the visible light image; acquiring a first absolute phase by using the first three-dimensional point cloud and the wrapping phase; Reconstructing a second three-dimensional point cloud of the object according to the modulated fringe image and the first absolute phase; the obtaining a first absolute phase by using the first three-dimensional point cloud and the wrapped phase includes: acquiring a first calibration parameter of an infrared projector; solving a second absolute phase by using the first three-dimensional point cloud and the first calibration parameter; And solving the first absolute phase by using the second absolute phase and the wrapping phase.
2. The method of three-dimensional reconstruction according to claim 1, wherein, The obtaining a modulation stripe image includes: projecting N-step phase shift stripes on the surface of the object by using an infrared projector, wherein N is an integer greater than 0; and acquiring a modulation fringe image of the phase-shift fringe modulated by the surface of the object.
3. The method of three-dimensional reconstruction according to claim 1, wherein, The obtaining a first three-dimensional point cloud by using the infrared image and the visible light image includes: acquiring a first space invariant feature of the infrared image; acquiring a second space-invariant feature of the visible light image; establishing a disparity map based on the first spatially invariant feature and the second spatially invariant feature; and converting the parallax map into the first three-dimensional point cloud by using the infrared camera calibration parameters and the visible light calibration parameters.
4. The method for three-dimensional reconstruction according to claim 3, wherein, The establishing a disparity map based on the first spatially invariant feature and the second spatially invariant feature includes: inputting the first space invariant feature and the second space invariant feature into a stereo matching network, and outputting the parallax map through the stereo matching network; The three-dimensional matching network is obtained by carrying out weighted training by using a smooth loss function and an L1 loss function.
5. The method of three-dimensional reconstruction according to claim 1, wherein, After the first three-dimensional point cloud is acquired by using the infrared image and the visible light image, the three-dimensional reconstruction method further comprises the following steps: determining a measurement range of an infrared camera based on the fringe period spatial span of the modulated fringe image; acquiring an offset value based on the measurement range and the actual depth of the object; and shifting the first three-dimensional point cloud along a preset direction according to the shift value to obtain a third three-dimensional point cloud.
6. The method of three-dimensional reconstruction according to claim 1, wherein, Said reconstructing a second three-dimensional point cloud of the object from said modulated fringe image and said first absolute phase, comprising: acquiring a second calibration parameter of the infrared camera; constructing a first equation by using the second calibration parameters and the pixel coordinates of the modulation fringe image; Constructing a second equation by using the first calibration parameter and the pixel value of the modulation fringe image; constructing a third equation using the pixel values of the modulated fringe image and the first absolute phase; and carrying out joint solution on the first equation, the second equation and the third equation to obtain a second three-dimensional point cloud of the object.
7. The method of three-dimensional reconstruction according to claim 6, wherein, The constructing a third equation using the pixel values of the modulation fringe image and the first absolute phase includes: Acquiring a third calibration parameter of the infrared projector, wherein the first calibration parameter comprises a projection matrix, and the third calibration parameter comprises a transverse projection fringe frequency, a longitudinal projection fringe frequency, a projector transverse resolution and/or a projector longitudinal resolution; And constructing the third programming by using the pixel value of the modulation fringe image, the first absolute phase and the third calibration parameter.
8. A three-dimensional reconstruction device, characterized in that it comprises a processor and a memory, in which program data are stored, the processor being adapted to execute the program data for implementing the three-dimensional reconstruction method according to any one of claims 1-7.
9. A computer readable storage medium for storing program data, which when executed by a processor is adapted to carry out the three-dimensional reconstruction method according to any one of claims 1-7.

Description

Three-dimensional reconstruction method, three-dimensional reconstruction device, and computer-readable storage medium Technical Field The present application relates to the field of computer vision, and in particular, to a three-dimensional reconstruction method, a three-dimensional reconstruction device, and a computer readable storage medium. Background Three-dimensional reconstruction is a fundamental technology in computer vision, and is widely applied to the fields of robotics, autopilot, industrial detection, AR, somatosensory entertainment and the like. In the current mainstream three-dimensional measurement technology, the time-of-flight technology has the advantages of high measurement speed, high equipment cost, large volume and low precision, the binocular stereoscopic vision technology has the advantages of high measurement speed, limited precision and easiness in being influenced by illumination, textures and the like, and the structured light measurement technology has the advantages of high precision and high spot density, wherein the speckle structured light technology only needs one projection, but can not obtain dense three-dimensional data. The three-dimensional data obtained by fringe projection profilometry is dense, but in order to achieve high accuracy, it is often necessary to project multiple encoding patterns, and the measurement speed is greatly limited. Consumer-level depth cameras such as KinectV a 1, the faceid module of the iPhone is usually composed of an infrared projector, an infrared camera and a visible light camera, the infrared projector projects an infrared coded image to a target, the infrared camera shoots the coded image modulated by the shape of an object so as to reconstruct three dimensions, the visible light camera is used for collecting color images and does not participate in three dimension reconstruction, thereby wasting sensors and information is caused, and the precision is limited to a certain extent. Disclosure of Invention The application provides a three-dimensional reconstruction method, a three-dimensional reconstruction device and a computer readable storage medium. The application provides a three-dimensional reconstruction method, which comprises the following steps: Acquiring a modulation fringe image, and acquiring a wrapping phase based on the modulation fringe image; Acquiring an infrared image based on the modulated stripe image; Acquiring a first three-dimensional point cloud by utilizing the infrared image and the visible light image; acquiring a first absolute phase by using the first three-dimensional point cloud and the wrapping phase; Reconstructing a second three-dimensional point cloud of the object according to the modulated fringe image and the first absolute phase. Wherein, the obtaining the modulation stripe image includes: projecting N-step phase shift stripes on the surface of the object by using an infrared projector, wherein N is an integer greater than 0; and acquiring a modulation fringe image of the phase-shift fringe modulated by the surface of the object. The acquiring a first three-dimensional point cloud by using the infrared image and the visible light image comprises the following steps: acquiring a first space invariant feature of the infrared image; acquiring a second space-invariant feature of the visible light image; establishing a disparity map based on the first spatially invariant feature and the second spatially invariant feature; and converting the parallax map into the first three-dimensional point cloud by using the infrared camera calibration parameters and the visible light calibration parameters. Wherein the establishing a disparity map based on the first spatially invariant feature and the second spatially invariant feature comprises: inputting the first space invariant feature and the second space invariant feature into a stereo matching network, and outputting the parallax map through the stereo matching network; The three-dimensional matching network is obtained by carrying out weighted training by using a smooth loss function and an L1 loss function. Wherein, after the first three-dimensional point cloud is obtained by using the infrared image and the visible light image, the three-dimensional reconstruction method further comprises: determining a measurement range of an infrared camera based on the fringe period spatial span of the modulated fringe image; acquiring an offset value based on the measurement range and the actual depth of the object; and shifting the first three-dimensional point cloud along a preset direction according to the shift value to obtain a third three-dimensional point cloud. The obtaining a first absolute phase by using the first three-dimensional point cloud and the wrapped phase includes: acquiring a first calibration parameter of an infrared projector; solving a second absolute phase by using the first three-dimensional point cloud and the first calibration parameter; A