CN-121999108-A - Three-dimensional Gaussian splatter training method, system, equipment and medium

CN121999108ACN 121999108 ACN121999108 ACN 121999108ACN-121999108-A

Abstract

The application provides a three-dimensional Gaussian splatter training method, a system, equipment and a medium, wherein the three-dimensional Gaussian splatter training method comprises the steps of projecting three-dimensional Gaussian to be processed to a two-dimensional image plane so as to be divided into sub-image blocks; the method comprises the steps of predicting task load time of a global sub-block according to executed sub-block execution time to obtain real-time task load information, dynamically scheduling based on the task load information to determine execution sequence of any sub-block, and carrying out reduction processing on internal processes of a streaming multiprocessor and merging processing among the streaming multiprocessor layers in the processing process of the sub-block by the streaming multiprocessor. The three-dimensional Gaussian splatter training method can solve the problem of unbalanced task load and improve the processing efficiency of three-dimensional Gaussian splatter stages.

Inventors

NIU GENG
LIANG XIAOYAO
JING NAIFENG
LIU ZIZHAO
LI JIAMING
WEN ZHICHENG
KONG DEHUI
XU KE

Assignees

上海交通大学
深圳市中兴微电子技术有限公司

Dates

Publication Date: 20260508
Application Date: 20251212

Claims (10)

1. A three-dimensional gaussian splatter training method, comprising: projecting the three-dimensional Gaussian to be processed to a two-dimensional image plane to be divided into sub-image blocks; predicting the task load time of the global sub-block according to the execution time of the executed sub-block to acquire real-time task load information; Dynamically scheduling based on the task load information to determine the execution sequence of any sub-block; and in the process of processing the sub-image blocks by the streaming multiprocessor, performing reduction processing on the internal process of the streaming multiprocessor, and performing merging processing on the interlayer of the streaming multiprocessor.
2. The three-dimensional gaussian splatter training method of claim 1, wherein predicting a task load time of a global sub-tile based on an execution time of an executed sub-tile to obtain real-time task load information comprises: Processing any sub-image block to obtain an executed sub-image block; predicting the execution time of surrounding unexecuted sub-blocks by taking the executed sub-blocks as the center to update the actual execution time of the surrounding unexecuted sub-blocks; And processing based on the actual execution time of the surrounding unexecuted sub-blocks, and predicting sub-blocks without the acquired actual execution time layer by layer to acquire the actual execution time of each sub-block as the task load information.
3. The three-dimensional gaussian splatter training method of claim 1, wherein dynamically scheduling to determine an execution order of any sub-tile based on the task load information comprises: Selecting a first batch of sub-blocks at intervals as initial sub-blocks to be processed; predicting the execution time of the peripheral non-executed sub-blocks by taking the initial sub-block as the center so as to update the actual execution time of the peripheral non-executed sub-blocks; Screening out a second batch of sub-blocks along the diagonal direction from the surrounding sub-blocks which are not executed for processing; predicting execution times of remaining unexecuted sub-tiles based on the initial sub-tiles and the second batch of sub-tiles that have been executed to update actual execution times of the remaining unexecuted sub-tiles; And processing the residual unexecuted sub-blocks according to the actual execution time of the residual unexecuted sub-blocks.
4. The three-dimensional gaussian splatter training method of claim 3, wherein the processing of the first batch of sub-blocks as initial sub-blocks at intervals further comprises: the initial sub-tiles are distributed into different streaming multiprocessors in a round robin fashion.
5. The three-dimensional gaussian splatter training method of claim 1, wherein dynamically scheduling to determine an execution order of any sub-tile based on the task load information further comprises: And sequencing according to the task quantity of each sub-block, and executing each sub-block from high to low in sequence.
6. The three-dimensional gaussian splatter training method of claim 1, wherein during processing of said sub-graph blocks by a streaming multiprocessor, performing reduction processing on internal processes of said streaming multiprocessor and performing merging processing on layers of said streaming multiprocessor comprises: performing reduction processing on atomic operations pointing to the same Gaussian so as to restrict the internal thread bundles of the streaming multiprocessor; utilizing a variable addition tree to dynamically calculate paths according to input data, and carrying out reduction processing on a plurality of data streams pointing to different gaussians; and temporarily storing the atomic update requests processed by the variable addition tree, and carrying out merging processing among the streaming multiprocessor layers.
7. The three-dimensional gaussian splatter training method of claim 6, further comprising: the atomic operations of different sub-blocks executed by the same stream type multiprocessor are combined.
8. A three-dimensional gaussian splatter training system, comprising: The two-dimensional segmentation module is used for projecting the three-dimensional Gaussian to be processed to a two-dimensional image plane so as to divide the three-dimensional Gaussian to be processed into sub-image blocks; The load prediction module is used for predicting the task load time of the global sub-block according to the execution time of the executed sub-block so as to acquire real-time task load information; The task scheduling module is used for dynamically scheduling based on the task load information so as to determine the execution sequence of any sub-block; And the atomic protocol module is used for carrying out reduction processing on the internal process of the streaming multiprocessor and carrying out merging processing on the interlayer of the streaming multiprocessor in the processing process of the streaming multiprocessor on any sub-block.
9. An electronic device, the electronic device comprising: A memory for storing a computer program; a processor for executing the computer program stored by the memory to cause the electronic device to perform the three-dimensional gaussian splatter training method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the three-dimensional gaussian splatter training method of any of claims 1 to 7.

Description

Three-dimensional Gaussian splatter training method, system, equipment and medium Technical Field The application belongs to the technical field of three-dimensional reconstruction, relates to a three-dimensional Gaussian splash training method, and particularly relates to a three-dimensional Gaussian splash training method, a system, equipment and a medium. Background In recent years, three-dimensional scene reconstruction technology has made a revolutionary breakthrough due to deep learning and computer graphics's deep fusion. In this field, 3D gaussian splatter (3 DGaussian Splatting, abbreviated as 3 DGS) is an emerging explicit modeling method, and by virtue of its efficient real-time rendering capability and excellent visual quality, it is rapidly becoming a focus of attention in academia and industry. Compared with the traditional implicit characterization method (such as NeRF), the 3DGS abandons the implicit coding mode relying on a multi-layer perceptron (MLP), and instead adopts an explicit 3D Gaussian distribution to directly model a scene. Each gaussian series of parameters describes its geometric and appearance characteristics and enables efficient projection from 3D to 2D through differentiable rasterization. The design not only avoids NeRF lengthy volume rendering calculations, but also supports dynamic adjustment of gaussian density to optimize reconstruction accuracy, which makes it exhibit significant advantages in complex scene reconstruction. However, with the expansion of the application scene of 3DGS, the bottleneck problem of training efficiency is increasingly prominent, and further development is restricted. Disclosure of Invention The application provides a three-dimensional Gaussian splatter training method, a system, equipment and a medium, which are used for solving the problem of low three-dimensional Gaussian splatter training efficiency in the prior art. In a first aspect, the present application provides a three-dimensional gaussian splatter training method. The three-dimensional Gaussian splatter training method comprises the steps of projecting a three-dimensional Gaussian to be processed to a two-dimensional image plane to be divided into sub-blocks, predicting task load time of a global sub-block according to execution time of the executed sub-blocks to obtain real-time task load information, dynamically scheduling based on the task load information to determine execution sequence of any sub-block, and carrying out reduction processing on internal processes of a streaming multiprocessor and merging processing among the streaming multiprocessor layers in the processing process of the sub-blocks by the streaming multiprocessor. In the application, the task load time of the global sub-block is predicted in the forward stage of three-dimensional Gaussian splatter, and the dynamic scheduling is carried out based on the real-time task load information. The internal process is reduced in the back propagation stage, and the interlayer is subjected to merging treatment. The three-dimensional Gaussian splatter training method can solve the problem of unbalanced task load, break the atomic operation bottleneck of the counter-propagation stage and improve the processing efficiency of the three-dimensional Gaussian splatter stage. In one implementation manner of the first aspect, predicting the task load time of the global sub-block according to the executed sub-block execution time to obtain real-time task load information includes processing any sub-block to obtain the executed sub-block, predicting the execution time of surrounding non-executed sub-blocks with the executed sub-block as a center to update the actual execution time of the surrounding non-executed sub-blocks, processing based on the actual execution time of the surrounding non-executed sub-blocks, and predicting sub-blocks without obtaining the actual execution time layer by layer to obtain the actual execution time of each sub-block as the task load information. In one implementation manner of the first aspect, the task load information is dynamically scheduled to determine an execution sequence of any sub-block, wherein the execution sequence comprises selecting a first batch of sub-blocks as an initial sub-block at intervals to be processed, predicting execution time of surrounding non-executed sub-blocks with the initial sub-block as a center to update actual execution time of the surrounding non-executed sub-blocks, screening a second batch of sub-blocks along a diagonal direction from the surrounding non-executed sub-blocks to be processed, predicting execution time of a residual non-executed sub-block based on the executed initial sub-block and the second batch of sub-blocks to update actual execution time of the residual non-executed sub-block, and processing the residual non-executed sub-block according to the actual execution time of the residual non-executed sub-block. In one implementation of the first as