CN-121392155-B - Three-dimensional Gaussian representation generation method, device and equipment based on scene data

CN121392155BCN 121392155 BCN121392155 BCN 121392155BCN-121392155-B

Abstract

The disclosure provides a three-dimensional Gaussian representation generation method, device and equipment based on scene data, relates to the technical field of computers, and particularly relates to the technical field of three-dimensional Gaussian representation generation and three-dimensional modeling. The method comprises the steps of generating scene point clouds based on triangular grids corresponding to models in three-dimensional scene data, generating camera poses based on house type structures in the three-dimensional scene data, generating rendering images based on the camera poses and the three-dimensional scene data, and generating three-dimensional Gaussian representations based on the scene point clouds, the camera poses and the rendering images.

Inventors

ZHOU YUELING
FU YITONG
LI LINGLING
HUANG XIAOHUANG

Assignees

杭州群核信息技术有限公司

Dates

Publication Date: 20260508
Application Date: 20251216

Claims (13)

1. A method of generating a three-dimensional gaussian representation based on scene data, comprising: Generating scene point clouds based on triangular grids corresponding to a model in three-dimensional scene data, wherein the scene point clouds comprise triangular grids corresponding to the model and a three-dimensional transformation matrix, wherein the triangular grids corresponding to the model are obtained; Generating a camera pose based on the house type structure in the three-dimensional scene data; Generating a rendered image based on the camera pose and the three-dimensional scene data; generating a three-dimensional gaussian representation based on the scene point cloud, the camera pose, and the rendered image; Sampling based on the scaled triangular mesh to obtain a scene point cloud, wherein the sampling comprises the following steps: according to the surface area of the scaled triangular mesh and the given point spacing, uniformly sampling the surface of the scaled triangular mesh to obtain a plurality of basic points; calculating according to the rotation parameters and/or the translation parameters in the three-dimensional change matrix based on the plurality of basic points to obtain a plurality of newly added points; And obtaining point clouds of scaling groups corresponding to the scaling values based on the plurality of basic points and the plurality of newly added points, wherein the point clouds of all scaling groups of one model form the point clouds of the model, and the point clouds of all models in the three-dimensional scene data form the scene point clouds.
2. The method of claim 1, wherein generating a camera pose based on a house type structure in the three-dimensional scene data comprises: acquiring the number of camera points required by a room based on the camera distance and the volume of the room in the house type structure; poisson sampling is carried out in the room, and camera point positions in the room are determined; and generating a camera view angle for each camera point to obtain the camera pose.
3. The method of claim 1, wherein generating a camera pose based on a house type structure in the three-dimensional scene data comprises: acquiring an annular curve of a room based on a plane contour line of the room in the house type structure; copying the annular curve based on the height of the room to obtain a multi-layer annular curve; determining camera points required by the room on the multi-layer annular curve based on camera spacing; and determining the camera view angle of each camera point position on the layer of annular curve based on the curve direction of each layer of annular curve to obtain the camera pose.
4. A method according to any one of claims 1 to 3, generating a rendered image based on the camera pose and the three-dimensional scene data, comprising: Synthesizing an animation sequence of the camera based on the plurality of camera poses; Constructing a scene to be rendered based on the animation sequence and the three-dimensional scene data; And rendering the scene to be rendered frame by using a rendering engine to obtain a rendering image corresponding to each camera pose.
5. The method of any of claims 1-3, wherein generating a three-dimensional gaussian representation based on the scene point cloud, the camera pose, and the rendered image comprises: and determining an initial position by using the scene point cloud, determining a view angle of a predicted image by using the camera pose, and generating a three-dimensional Gaussian representation by using the rendered image as an optimization target image based on a three-dimensional Gaussian sputtering algorithm.
6. A three-dimensional gaussian representation generating apparatus based on scene data, comprising: The system comprises a three-dimensional scene data acquisition module, a point cloud generation module, a scaling submodule, a point cloud submodule and a point cloud processing module, wherein the three-dimensional scene data acquisition module is used for acquiring a three-dimensional scene data; the pose generation module is used for generating a camera pose based on the house type structure in the three-dimensional scene data; An image generation module for generating a rendered image based on the camera pose and the three-dimensional scene data; a gaussian generation module for generating a three-dimensional gaussian representation based on the scene point cloud, the camera pose, and the rendered image; The method comprises the steps of obtaining a scaling value, obtaining a plurality of basic points, obtaining a plurality of newly added points by calculating according to rotation parameters and/or translation parameters in a three-dimensional change matrix based on the basic points, obtaining a scaling group point cloud corresponding to the scaling value based on the basic points and the newly added points, wherein the scaling group point cloud of one model forms the model point cloud, and the scene point cloud is formed by the point clouds of all models in three-dimensional scene data.
7. The apparatus of claim 6, wherein the pose generation module comprises: The first acquisition submodule is used for acquiring the number of camera points needed by a room based on the camera distance and the volume of the room in the house type structure; the first determining submodule is used for carrying out poisson sampling in the room and determining camera points in the room; and the first pose sub-module is used for generating a camera view angle for each camera point position to obtain the camera pose.
8. The apparatus of claim 6, wherein the pose generation module comprises: The second acquisition submodule is used for acquiring an annular curve of a room based on a plane contour line of the room in the house type structure; The replication submodule is used for replicating the annular curve based on the height of the room to obtain a multi-layer annular curve; A second determining submodule, configured to determine a camera point location required by the room on the multi-layer annular curve based on a camera pitch; And the second pose sub-module is used for determining the camera view angle of each camera point position on the layer of annular curve based on the curve direction of each layer of annular curve to obtain the camera pose.
9. The apparatus of any of claims 6 to 8, the image generation module comprising: a synthesis submodule for synthesizing an animation sequence of the camera based on the plurality of camera poses; a construction sub-module for constructing a scene to be rendered based on the animation sequence and the three-dimensional scene data; and the rendering sub-module is used for rendering the scene to be rendered frame by using a rendering engine to obtain a rendering image corresponding to each camera pose.
10. The apparatus of any of claims 6 to 8, wherein the gaussian generation module is to determine an initial position using the scene point cloud, determine a perspective of a predicted image using the camera pose, generate a three-dimensional gaussian representation based on a three-dimensional gaussian sputtering algorithm using the rendered image as an optimization target image.
11. An electronic device, comprising: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
12. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-5.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-5.

Description

Three-dimensional Gaussian representation generation method, device and equipment based on scene data Technical Field The present disclosure relates to the field of computer technology, and in particular, to the field of three-dimensional gaussian representation generation and three-dimensional modeling. Background Three-dimensional scene generation and reconstruction mainly depend on manual reconstruction, hardware reconstruction and the like. The manual reconstruction manually constructs the scene and the camera position through three-dimensional modeling software, and generates a rendered image, so that the modeling efficiency is low. And acquiring point cloud data by utilizing hardware modes such as structured light, laser scanning and the like, and reconstructing a three-dimensional scene by combining traditional multi-view geometry and other means. The consistency of training data of hardware reconstruction is not strong, a large error exists in a low texture area, and the corresponding relation of an image, point cloud and a camera is difficult to accurately establish, so that the training precision is influenced. Disclosure of Invention The present disclosure provides a method, apparatus, and device for generating a three-dimensional gaussian representation based on scene data to solve or mitigate one or more technical problems in the prior art. In a first aspect, the present disclosure provides a method for generating a three-dimensional gaussian representation based on scene data, including: generating scene point cloud based on triangular grids corresponding to a model in three-dimensional scene data, wherein the scene point cloud comprises triangular grids corresponding to the model and a three-dimensional transformation matrix, scaling the triangular grids corresponding to the model based on scaling values of the three-dimensional transformation matrix, sampling based on the scaled triangular grids, and obtaining the scene point cloud; Generating a camera pose based on the house type structure in the three-dimensional scene data; Generating a rendered image based on the camera pose and the three-dimensional scene data; a three-dimensional gaussian representation is generated based on the scene point cloud, the camera pose, and the rendered image. In a second aspect, the present disclosure provides a three-dimensional gaussian representation generating apparatus based on scene data, comprising: The system comprises a three-dimensional scene data acquisition module, a point cloud generation module and a scene point cloud generation module, wherein the three-dimensional scene data acquisition module is used for acquiring a three-dimensional grid corresponding to the model and a three-dimensional transformation matrix; the pose generation module is used for generating a camera pose based on the house type structure in the three-dimensional scene data; An image generation module for generating a rendered image based on the camera pose and the three-dimensional scene data; And the Gaussian generation module is used for generating a three-dimensional Gaussian representation based on the scene point cloud, the camera pose and the rendered image. In a third aspect, an electronic device is provided, comprising: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the embodiments of the present disclosure. In a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are for causing the computer to perform a method according to any one of the embodiments of the present disclosure. In a fifth aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method according to any of the embodiments of the present disclosure. It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification. Drawings In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments provided according to the disclosure and are not to be considered limiting of its scope. FIG. 1 is a flow diagram of a method of generating a three-dimensional Gaussian representation based on scene data according to an embodiment of the disclosure; FIG. 2 is a flow diagram of a method of generating a three-dimensional Gaussian representation based on scene data according t