CN-121982257-A - Three-dimensional reconstruction method, three-dimensional reconstruction device, storage medium, program product, and computer device

CN121982257ACN 121982257 ACN121982257 ACN 121982257ACN-121982257-A

Abstract

The application discloses a three-dimensional reconstruction method, a device, a storage medium, a program product and computer equipment, wherein the method comprises the steps of determining corresponding back view images on the basis of acquiring front view images, namely, acquiring images on multiple view angles is not required, further improving the efficiency and the application range of three-dimensional reconstruction, taking image features and normal line features of each image as priori constraints to carry out reconstruction, combining random sampling points of an object in a three-dimensional space to determine depth hidden functions corresponding to the object, and utilizing the depth hidden functions to assist the front view images and the back view images to carry out three-dimensional reconstruction processing, thereby improving the precision of three-dimensional reconstruction.

Inventors

ZHOU YIFAN
SHEN WEI
LI KEKE
YANG XIAOMING

Assignees

中移(苏州)软件技术有限公司
中国移动通信集团有限公司

Dates

Publication Date: 20260505
Application Date: 20260128

Claims (10)

1. A three-dimensional reconstruction method, comprising: Acquiring a front view image, wherein the front view image corresponds to an object to be reconstructed; Determining a rear view image corresponding to the front view image; determining respective image features of the front view image and the rear view image, and determining respective normal features of the front view image and the rear view image; Determining a depth hidden function corresponding to the object based on the image feature, the normal feature and random sampling points of the object in a three-dimensional space; and carrying out three-dimensional reconstruction processing based on the front view image, the rear view image and the depth hidden function, and generating a three-dimensional model corresponding to the object.
2. The method of claim 1, wherein the determining a rear view image corresponding to the front view image comprises: Determining two-dimensional contour information corresponding to the front view image; and inputting the front view image and the two-dimensional contour information into an image conversion network to obtain the rear view image output by the image conversion network, wherein the image conversion network is constructed based on a condition generation countermeasure algorithm.
3. The method of claim 1, wherein the determining a depth hidden function corresponding to the object based on the image feature, the normal feature, and random sampling points of the object in three-dimensional space comprises: Adding Gaussian disturbance to the random sampling points to obtain target sampling points; And taking the image features and the normal features as priori constraints of a depth hidden function, and constructing the depth hidden function according to the target sampling points.
4. The method of claim 1, wherein the three-dimensional reconstruction process includes a surface reconstruction and a texture domain reconstruction; The three-dimensional reconstruction processing is performed based on the front view image, the rear view image and the depth hidden function, and a three-dimensional model corresponding to the object is generated, including: Calling a multi-layer perceptron to obtain the volume occupancy probability of the three-dimensional point corresponding to the object based on the depth hidden function; performing surface reconstruction based on the three-dimensional point volume occupation probability to generate an initial three-dimensional model corresponding to the object; and reconstructing a texture domain based on the front view image, the rear view image and the initial three-dimensional model to generate the three-dimensional model.
5. The method of claim 4, wherein the generating the three-dimensional model based on texture domain reconstruction of the front view image, the rear view image, and the initial three-dimensional model comprises: determining each grid surface of the object; determining a view corresponding to each grid surface based on a normal vector corresponding to each grid surface and a camera ray vector corresponding to the front view image; And reconstructing texture domains of the grid planes based on the views corresponding to the grid planes and the front view image and the rear view image so as to generate the three-dimensional model.
6. The method of claim 5, wherein the view corresponding to each grid surface is a front view, a back view, or a boundary view; The reconstructing the texture domain of each grid surface based on the view corresponding to each grid surface, the front view image and the rear view image includes: For each of the mesh faces, Under the condition that the corresponding view is a front view, reconstructing a texture domain of the grid surface based on the front view image; under the condition that the corresponding view is a rear view, reconstructing a texture domain of the grid surface based on the rear view image; And under the condition that the corresponding view is a boundary view, reconstructing a texture domain of the grid surface based on the front view image and the rear view image.
7. A three-dimensional reconstruction apparatus, comprising: The front view acquisition module is used for acquiring a front view image, wherein the front view image corresponds to an object to be reconstructed; a rear view determining module, configured to determine a rear view image corresponding to the front view image; The feature determining module is used for determining respective image features of the front view image and the rear view image and determining respective normal features of the front view image and the rear view image; the hidden function calculation module is used for determining a depth hidden function corresponding to the object based on the image feature, the normal feature and random sampling points of the object in a three-dimensional space; and the reconstruction module is used for carrying out three-dimensional reconstruction processing based on the front view image, the rear view image and the depth hidden function and generating a three-dimensional model corresponding to the object.
8. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the method of any of claims 1-6.
9. A computer program product comprising computer instructions which, when executed by a processor, implement the method of any of claims 1-6.
10. A computer device comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the method of any of claims 1-6 when the computer program is executed.

Description

Three-dimensional reconstruction method, three-dimensional reconstruction device, storage medium, program product, and computer device Technical Field The present application relates to the field of image processing technologies, and in particular, to a three-dimensional reconstruction method, apparatus, storage medium, program product, and computer device. Background Three-dimensional reconstruction techniques aim to recover a three-dimensional geometric model from two-dimensional image information. Compared with a two-dimensional image, the three-dimensional model contains more parameter information, brings more stereoscopic impression, and is easier to bring visual impact to a user. Compared with manual modeling, the three-dimensional reconstruction technology can greatly reduce cost and can support real-time modeling. In the related art, a multi-view three-dimensional reconstruction method may be used, for example, by calibrating and aligning acquired multi-view two-dimensional images, extracting object feature points from the aligned two-dimensional images, and converting the object feature points into three-dimensional spatial feature points through a computer vision algorithm. However, the three-dimensional reconstruction technology with multiple view angles requires shooting a plurality of two-dimensional images with different angles on the object, so that on one hand, time and effort are consumed, and further, the reconstruction efficiency is low, on the other hand, the acquired angle is difficult to accurately control, so that the key point characteristics acquired in the acquisition process are not accurate enough, and further, the reconstruction precision is low. In addition, in many application scenes, it is difficult to acquire object images of multiple views in real time, so that three-dimensional reconstruction cannot be completed, and the application range of the three-dimensional reconstruction is narrow. Disclosure of Invention In order to solve the technical problems, embodiments of the present application provide a three-dimensional reconstruction method, apparatus, storage medium, program product, and computer device, which can improve efficiency, accuracy, and application range of three-dimensional reconstruction. In a first aspect, an embodiment of the present application provides a three-dimensional reconstruction method, including: Acquiring a front view image, wherein the front view image corresponds to an object to be reconstructed; Determining a rear view image corresponding to the front view image; determining respective image features of the front view image and the rear view image, and determining respective normal features of the front view image and the rear view image; Determining a depth hidden function corresponding to the object based on the image feature, the normal feature and random sampling points of the object in a three-dimensional space; and carrying out three-dimensional reconstruction processing based on the front view image, the rear view image and the depth hidden function, and generating a three-dimensional model corresponding to the object. Optionally, the determining the rear view image corresponding to the front view image includes: Determining two-dimensional contour information corresponding to the front view image; and inputting the front view image and the two-dimensional contour information into an image conversion network to obtain the rear view image output by the image conversion network, wherein the image conversion network is constructed based on a condition generation countermeasure algorithm. Optionally, the determining a depth hidden function corresponding to the object based on the image feature, the normal feature and the random sampling point of the object in the three-dimensional space includes: Adding Gaussian disturbance to the random sampling points to obtain target sampling points; And taking the image features and the normal features as prior constraints of a depth hidden function, taking the image features and the normal features as prior constraints of the depth hidden function, and constructing the depth hidden function according to the target sampling points. Optionally, the three-dimensional reconstruction process includes a surface reconstruction and a texture domain reconstruction; The three-dimensional reconstruction processing is performed based on the front view image, the rear view image and the depth hidden function, and a three-dimensional model corresponding to the object is generated, including: Calling a multi-layer perceptron to obtain the volume occupancy probability of the three-dimensional point corresponding to the object based on the depth hidden function; performing surface reconstruction based on the three-dimensional point volume occupation probability to generate an initial three-dimensional model corresponding to the object; and reconstructing a texture domain based on the front view image, the rear view im