Search

CN-122023680-A - Low-computation-force rapid three-dimensional modeling method and system based on 3DGS

CN122023680ACN 122023680 ACN122023680 ACN 122023680ACN-122023680-A

Abstract

The invention relates to the technical field of three-dimensional modeling, in particular to a low-computation-force rapid three-dimensional modeling method and system based on 3 DGS. And carrying out robust aggregation on the residual errors according to preset tiles serving as statistical units, constructing a projection statistical spectrum by combining Gaussian projection contribution, obtaining structural variation through period-crossing comparison, and forming complexity scores through residual error gating to divide a complex region and a flat region. And establishing tile-to-Gaussian object mapping according to the density scheduling state, directionally splitting and encrypting a complex region, absorbing, merging and sparsifying a middle region limited fine tuning appearance and transparency and a flat region, so that rapid convergence and stable modeling are realized under the condition of limited calculation force, and invalid calculation proportion is obviously reduced while reconstruction precision and structural stability are maintained.

Inventors

  • Cui Ningfeng
  • LEI XIAODONG

Assignees

  • 上海艾涛信息科技发展有限公司

Dates

Publication Date
20260512
Application Date
20260415

Claims (10)

  1. 1. The low-computation-power rapid three-dimensional modeling method based on 3DGS is applied to a graphic processing unit, and is characterized in that a current Gaussian field model is configured in the graphic processing unit, the current Gaussian field model is an updated Gaussian field model of the graphic processing unit in a last modeling period, each modeling period corresponds to image data of one view angle of an object to be modeled, and the method comprises the following steps: Acquiring image data of an object to be modeled in a current modeling period; Performing forward rendering based on the image data and the current Gaussian field model to obtain a rendered image, and determining residual information based on the rendered image and the image data; Determining complexity information based on the residual information and a Gaussian object set corresponding to the current Gaussian field model, wherein the complexity information is used for representing local geometric detail complexity of an object to be modeled in a current modeling period and dividing a complex area and a flat area, and each Gaussian object in the Gaussian object set is at least associated with a spatial position parameter, a shape parameter, a transparency parameter and a color representation parameter; And performing self-adaptive Gaussian density distribution on the current Gaussian field model according to the complexity information to obtain an updated Gaussian field model of the current modeling period, and performing three-dimensional modeling according to the updated Gaussian field model.
  2. 2. The 3 DGS-based low-computation-force rapid three-dimensional modeling method according to claim 1, wherein if the current modeling period is the first modeling period, the process of constructing the current gaussian field model comprises: Acquiring image data of a view angle corresponding to a first modeling period, and executing feature extraction based on the image data to obtain a feature point set; Performing camera pose solving based on the feature point set to determine camera parameters of a first modeling period, and constructing an initial geometric reference according to the camera parameters; Generating an initial Gaussian object set based on the initial geometric reference, wherein the spatial position parameter of each Gaussian object in the initial Gaussian object set is determined by the initial geometric reference, the shape parameter is determined by a preset covariance initialization strategy, the transparency parameter is determined by a preset opacity initial value, and the color representation parameter is determined by the pixel color of the image data at a corresponding projection position.
  3. 3. The 3 DGS-based low-computational-force fast three-dimensional modeling method of claim 1, wherein performing forward rendering based on the image data and the current gaussian field model comprises: acquiring camera parameters corresponding to the image data in a current modeling period, and projecting each Gaussian object in the Gaussian object set from a three-dimensional space to an image plane based on the camera parameters to obtain projection ellipse parameters of each Gaussian object on the image plane; and sequencing the projection ellipses according to the depth information of each Gaussian object for the pixel points on the image plane, and performing Alpha mixing pixel by pixel based on the transparency parameter to generate the rendered image.
  4. 4. A 3 DGS-based low-computational-force fast three-dimensional modeling method according to claim 3, wherein the determination of residual information comprises: Determining a color contribution of each Gaussian object at a current viewing angle based on the color representation parameters in the process of generating the rendered image, wherein the color contribution at least comprises a spherical harmonic coefficient for representing a change of a viewing angle color; After the rendered image is generated, pixel-level residuals are calculated based on the rendered image and the image data, and the pixel-level residuals are aggregated through the color contributions to obtain the residual information.
  5. 5. The 3 DGS-based low-computational power fast three-dimensional modeling method of claim 1, wherein determining complexity information based on the set of gaussian objects for which the residual information corresponds to the current gaussian field model comprises: Aggregating residual information between the rendered image and the image data according to a preset tile dividing rule to obtain tile residual statistics of each tile, wherein the tile residual statistics at least comprise residual energy statistics and residual gradient statistics; Aggregating Gaussian projection contributions of each tile based on the Gaussian object set to obtain a projection statistical spectrum of each tile, wherein the projection statistical spectrum at least comprises coverage weight statistics, depth mixing dispersion statistics and anisotropic direction consistency statistics; Comparing the projection statistical spectrum of each tile in the current modeling period with the projection statistical spectrum of each tile in the previous modeling period to obtain the structural variation degree of each tile, and carrying out residual gating weighting on the structural variation degree based on the tile residual statistics to obtain the complexity score of each tile; And dividing a complex region and a flat region based on the complexity score, wherein the complex region comprises a tile corresponding region with the complexity score meeting a first threshold condition, the flat region comprises a tile corresponding region with the complexity score meeting a second threshold condition, and the first threshold is larger than the second threshold.
  6. 6. The 3 DGS-based low-power fast three-dimensional modeling method of claim 5, wherein aggregating residual information between the rendered image and the image data according to preset tile partitioning rules comprises: According to the residual information, combining the image data to calculate a pixel residual image; dividing the pixel residual map into a plurality of tile areas based on the preset tile dividing rule, wherein each tile area corresponds to a pixel set with a preset size; In each tile region, calculating pixel residual errors aiming at the pixel set, and carrying out robust aggregation on the pixel residual errors to obtain residual error energy statistics of the tile region, wherein the robust aggregation at least comprises one of mean aggregation, quantile aggregation and truncated mean aggregation; In each tile region, performing gradient operator calculation on the pixel residual errors to obtain residual error gradient diagrams, and aggregating the residual error gradient diagrams to obtain residual error gradient statistics of the tile region, wherein the residual error gradient statistics are used for representing spatial intensity of residual error change in the tile region; And weighting the residual energy statistic and the residual gradient statistic to obtain a tile residual statistic.
  7. 7. The 3 DGS-based low-computation-power rapid three-dimensional modeling method of claim 5, wherein aggregating gaussian projection contributions of each tile based on the set of gaussian objects to obtain a projection statistical spectrum of each tile comprises: determining target tile areas of all Gaussian objects; In each target tile area, accumulating the Gaussian object belonging to the target tile area based on transparency parameters and projected ellipse coverage weights to obtain coverage weight statistics; in each target tile area, based on the depth information of a Gaussian object belonging to the target tile area and the pixel-by-pixel Alpha mixing sequence, counting the effective mixing weight distribution of the Gaussian object in a preset depth hierarchy, and calculating the depth mixing dispersion statistic based on the effective mixing weight distribution; extracting the projection ellipse principal axis direction information of a Gaussian object belonging to the target tile area in each target tile area, and calculating the anisotropic direction consistency statistic based on the principal axis direction information; Combining the coverage weight statistic, the depth mixture dispersion statistic and the anisotropy direction consistency statistic into the projection statistic spectrum.
  8. 8. The 3 DGS-based low-computational power fast three-dimensional modeling method of claim 5, wherein performing adaptive gaussian density distribution on the current gaussian field model according to the complexity information comprises: determining a density scheduling state for each tile region based on the complexity score, wherein the density scheduling state at least comprises an encryption state, a fine tuning state and a sparse state; Establishing a mapping relation from the tile to the Gaussian object according to the density scheduling state to determine an encryption candidate Gaussian object set, a fine adjustment candidate Gaussian object set and a sparse candidate Gaussian object set, wherein the mapping relation is determined at least based on the overlapping degree of a projection ellipse of the Gaussian object and a tile area; Performing a directional splitting operation on the encrypted candidate Gaussian object set to generate at least one sub-Gaussian object, wherein the directional splitting operation comprises the steps of determining a splitting direction based on a residual gradient main direction and a projection ellipse main axis direction, and disturbing the spatial position parameter and the shape parameter in the splitting direction to obtain an initialization parameter of the sub-Gaussian object, and simultaneously enabling the sub-Gaussian object to inherit the prior values of the color representation parameter and the transparency parameter; performing parameter limited updating for the fine-tuning candidate Gaussian object set, wherein the parameter limited updating at least comprises updating a color representation parameter and a transparency parameter, and keeping a space position parameter and a shape parameter unchanged; And executing absorption merging operation on the sparse candidate Gaussian object set, wherein the absorption merging operation comprises the steps of determining Gaussian object clusters meeting similarity constraint in the same tile region, wherein the similarity constraint at least comprises color similarity constraint, principal axis direction similarity constraint and depth consistency constraint, merging the Gaussian object clusters into a merged Gaussian object, and carrying out weighted solution on spatial position parameters, shape parameters, transparency parameters and color representation parameters of the merged Gaussian object by the parameters of the Gaussian object clusters according to effective mixing weights.
  9. 9. The 3 DGS-based low-power fast three-dimensional modeling method of claim 8, wherein the encryption state corresponds to a region of tiles having a complexity score that satisfies the first threshold condition, the sparsity state corresponds to a region of tiles having a complexity score that satisfies the second threshold condition, and the fine tuning state corresponds to a region of tiles having a complexity score that is between the first threshold and the second threshold.
  10. 10. A 3 DGS-based low-computational-force rapid three-dimensional modeling system for implementing a 3 DGS-based low-computational-force rapid three-dimensional modeling method according to any one of claims 1 to 9, the system comprising: The period scheduling module is used for acquiring image data of an object to be modeled corresponding to the view angle of each modeling period, and maintaining a current Gaussian field model on the graphic processing unit, wherein the current Gaussian field model is an updated Gaussian field model of the previous modeling period; the complexity evaluation module is used for performing forward rendering on the basis of the image data and the current Gaussian field model to obtain a rendered image, determining residual information on the basis of the rendered image and the image data, and determining complexity information on the basis of a Gaussian object set corresponding to the residual information and the current Gaussian field model so as to divide a complex area and a flat area; And the modeling output module is used for executing self-adaptive Gaussian density distribution on the current Gaussian field model according to the complexity information to obtain an updated Gaussian field model of the current modeling period, and completing three-dimensional modeling output based on the updated Gaussian field model.

Description

Low-computation-force rapid three-dimensional modeling method and system based on 3DGS Technical Field The invention relates to the technical field of three-dimensional modeling, in particular to a low-computation-force rapid three-dimensional modeling method and system based on 3 DGS. Background The rapid reconstruction method based on three-dimensional Gaussian representation is widely applied to continuous view acquisition and online modeling scenes, and high-efficiency fitting of complex geometry and texture is achieved by arranging a large number of Gaussian objects in space and combining view-dependent color expression. However, in the actual engineering deployment process, especially on a mobile terminal or an embedded low-power device, the prior art generally adopts a global unified update strategy, that is, parameter optimization or density adjustment is performed on all gaussian objects in each modeling period, and the structural variation degree of different image areas is not distinguished. The processing mode is easy to cause two problems in a continuous modeling process, namely, on one hand, a flat or stabilized area is still repeatedly participated in high-overhead calculation to cause calculation power waste and increase of object number redundancy, and on the other hand, a local structure is rapidly changed or a region with frequently rearranged shielding relation lacks targeted encryption scheduling, so that residual errors are concentrated on a small number of Gaussian objects for a long time, and the situation of insufficient detail expression or reduced convergence speed occurs. In addition, when the existing method is used for splitting or merging objects, the existing method depends on a single residual error threshold value or object level statistic, lacks a structure change degree comparison mechanism crossing a modeling period, and is difficult to realize density self-adaptive distribution while maintaining model stability. Therefore, under the condition of low-calculation-force continuous modeling, how to realize differential updating of different areas, inhibit invalid calculation and improve model convergence efficiency on the premise of ensuring expression precision becomes a technical problem to be solved urgently. In order to solve the problems, the application designs a low-computation-force rapid three-dimensional modeling method and a system based on 3 DGS. Disclosure of Invention The invention aims to solve the technical problem of overcoming the defects of the prior art, and provides a 3 DGS-based low-computation-power rapid three-dimensional modeling method and a 3 DGS-based low-computation-power rapid three-dimensional modeling system. And carrying out robust aggregation on the residual errors according to preset tiles serving as statistical units, constructing a projection statistical spectrum by combining Gaussian projection contribution, obtaining structural variation through period-crossing comparison, and forming complexity scores through residual error gating to divide a complex region and a flat region. And establishing tile-to-Gaussian object mapping according to the density scheduling state, directionally splitting and encrypting a complex region, absorbing, merging and sparsifying a middle region limited fine tuning appearance and transparency and a flat region, and thus realizing rapid convergence and stable modeling under the condition of limited calculation force. In order to achieve the above purpose, the present invention provides the following technical solutions: The low-computation-force rapid three-dimensional modeling method based on 3DGS is applied to a graphic processing unit, wherein a current Gaussian field model is configured in the graphic processing unit, the current Gaussian field model is an updated Gaussian field model of the graphic processing unit in a last modeling period, each modeling period corresponds to image data of one view angle of an object to be modeled, and the method comprises the following steps: Acquiring image data of an object to be modeled in a current modeling period; Performing forward rendering based on the image data and the current Gaussian field model to obtain a rendered image, and determining residual information based on the rendered image and the image data; Determining complexity information based on the residual information and a Gaussian object set corresponding to the current Gaussian field model, wherein the complexity information is used for representing local geometric detail complexity of an object to be modeled in a current modeling period and dividing a complex area and a flat area, and each Gaussian object in the Gaussian object set is at least associated with a spatial position parameter, a shape parameter, a transparency parameter and a color representation parameter; And performing self-adaptive Gaussian density distribution on the current Gaussian field model according to the complexity informat