Search

CN-122023727-A - Lightweight packaging method and system applied to 3DGS model

CN122023727ACN 122023727 ACN122023727 ACN 122023727ACN-122023727-A

Abstract

The invention discloses a lightweight packaging method and system applied to a 3DGS model, which belong to the technical field of 3D modeling and packaging, and comprise the steps of decoupling Gaussian ellipsoid point cloud data streams of 3D Gaussian splatter modeling into geometric and appearance attribute streams, generating attribute data blocks through self-adaptive precision quantization coding, sorting and screening contribution degrees of ellipsoids to a rendered image to generate a multi-level sparse Gaussian ellipsoid set, carrying out staggered rearrangement and compression on the attribute streams through a parallel entropy coder to obtain compressed code streams, adding multi-resolution hierarchical indexes and incremental update metadata to the compressed code streams, and packaging the compressed code streams into a self-contained lightweight model package according to a network flow loading sequence.

Inventors

  • Cui Ningfeng
  • LEI XIAODONG

Assignees

  • 上海艾涛信息科技发展有限公司

Dates

Publication Date
20260512
Application Date
20260413

Claims (10)

  1. 1. The lightweight packaging method applied to the 3DGS model is characterized by comprising the following steps of: Receiving a Gaussian ellipsoid point cloud data stream generated by 3D Gaussian splats modeling, decoupling the Gaussian ellipsoid point cloud data stream into a geometric attribute stream and an appearance attribute stream, and respectively carrying out self-adaptive precision quantization coding on the geometric attribute stream and the appearance attribute stream to generate quantized attribute data blocks; taking the quantized attribute data blocks as input, and carrying out importance sorting and hierarchical screening on the contribution degree of the final rendered image according to each ellipsoid to generate a multi-level sparse Gaussian ellipsoid set, wherein each level of data set is a strict subset of the upper level; Inputting the multi-level sparse Gaussian ellipsoid set into a parallel entropy coder specially optimized for WebGPU memory layout, and performing interleaving rearrangement and compression on the geometric attribute stream and the appearance attribute stream by the parallel entropy coder according to the access mode of the GPU buffer region to output a compressed code stream; And adding multi-resolution level indexes and increment updating metadata for the compressed code stream, and packaging the compressed code stream into a self-contained lightweight model package according to the network stream loading sequence.
  2. 2. The lightweight packaging method applied to the 3DGS model of claim 1, wherein the receiving a gaussian ellipsoidal point cloud data stream generated by 3D gaussian splatter modeling, decoupling the gaussian ellipsoidal point cloud data stream into a geometric attribute stream and an appearance attribute stream, comprises: acquiring Gaussian ellipsoid point cloud data output after 3D Gaussian splatter modeling is completed, wherein each Gaussian ellipsoid represents a spatial point and has a group of attribute data; Extracting three-dimensional center point coordinates of all Gaussian ellipsoids in a three-dimensional scene, rotation quaternions representing the rotation state of the Gaussian ellipsoids in the three-dimensional space and three scaling coefficients representing the scaling degree of the Gaussian ellipsoids in the X-axis, Y-axis and Z-axis directions from the Gaussian ellipsoid point cloud data to jointly form a geometric attribute stream; And extracting multi-order spherical harmonic coefficients of all Gaussian ellipsoids for defining color appearance from the Gaussian ellipsoid point cloud data, and controlling the opacity value of the visibility of the Gaussian ellipsoids when light passes through to jointly form an appearance attribute stream.
  3. 3. The method for lightweight encapsulation applied to a 3DGS model according to claim 1, wherein the adaptively precision quantization encoding is performed on the geometric attribute stream and the appearance attribute stream, respectively, to generate quantized attribute data blocks, and the method comprises: Mapping the three-dimensional center point coordinates in the geometric attribute stream into a preset normalized numerical range by adopting a scene global bounding box, and converting floating point coordinates into integer indexes according to a preset geometric quantization bit number by using a scalar quantizer based on uniform quantization; performing quaternion normalization on the rotation quaternion in the geometric attribute stream, and mapping the continuous rotation quaternion into a discrete codebook index by searching elements with nearest Euclidean distance in a rotation codebook of each discrete direction of a pre-generated coverage unit sphere; Converting an original scaling value into a logarithmic space by adopting logarithmic domain conversion to three scaling coefficients in the geometric attribute stream to obtain a logarithmic scaling value, applying a scalar quantizer based on non-uniform quantization in the logarithmic space, and converting the logarithmic scaling value into an integer index according to a preset scaling quantization bit number; Calculating global statistical variances of all the spherical harmonic coefficients of all the gauss ellipsoids, dynamically distributing different quantization bit numbers according to the global statistical variances of the spherical harmonic coefficients of all the orders, and converting floating-point coefficients into integer indexes by using a scalar quantizer with uniform quantization according to the bit numbers distributed to each gauss ellipsoid; applying inverse function transformation of a Sigmoid function, mapping an opacity value in the appearance attribute stream to an unbounded real number domain, applying a scalar quantizer based on uniform quantization in the unbounded real number domain, and converting the transformed value into an integer index according to a preset opacity quantization bit number; Grouping and packaging all integer indexes generated in the quantization process according to the belonging Gaussian ellipsoid marks to form a structured quantized attribute data block.
  4. 4. The method of claim 1, wherein the performing importance ranking and hierarchical filtering on the contribution degree of the final rendered image according to each ellipsoid to generate a multi-level sparse gaussian ellipsoid set comprises: setting a target rendering viewpoint set, performing forward rasterization rendering simulation on each Gaussian ellipsoid based on the target rendering viewpoint set, and calculating the average opacity weight and the average color gradient amplitude of the pixel area covered by the corresponding Gaussian ellipsoid in each viewpoint image generated by simulation rendering; Calculating contribution degree scores of all Gaussian ellipsoids based on the weighted sum of the average opacity weight and the average color gradient amplitude, and sorting all Gaussian ellipsoids in a descending order according to the contribution degree scores to generate a sorted Gaussian ellipsoid list; Setting at least two sequentially reduced contribution degree score thresholds, screening out gauss ellipsoids with contribution degree scores higher than a first contribution degree score threshold based on a ordered gauss ellipsoid list to form a first-stage sparse gauss ellipsoid set, screening out gauss ellipses with contribution degree scores higher than a second contribution degree score threshold to form a second-stage sparse gauss ellipsoid set, and circularly analogizing to generate a multi-stage sparse gauss ellipsoid set, wherein the first-stage sparse gauss ellipsoid set is a strict subset of the second-stage sparse gauss ellipsoid set, and the second contribution degree score threshold is smaller than the first contribution degree score threshold.
  5. 5. The method for lightweight encapsulation applied to 3DGS model according to claim 1, wherein said parallel entropy encoder performs interleaving rearrangement and compression on the geometry attribute stream and the appearance attribute stream according to the access mode of the GPU buffer, and outputs a compressed code stream, comprising: An interleaved data storage structure is designed, wherein a single Gaussian ellipsoid is taken as a unit, a quantized center point coordinate X component integer value, a center point coordinate Y component integer value, a center point coordinate Z component integer value, a rotation quaternion X component integer value, a rotation quaternion Y component integer value, a rotation quaternion Z component integer value, a rotation quaternion W component integer value, a logarithmic quantization integer value of a scaling factor along an X axis, a logarithmic quantization integer value of a scaling factor along a Y axis, a logarithmic quantization integer value of a scaling factor along a Z axis and an anti-Sigmoid transformation quantization integer value of an opacity value are sequentially stored in a continuous storage space, and after the storage is completed, the quantization integer values of the corresponding Gaussian ellipsoid multi-order spherical harmonic coefficients are stored, and the spherical harmonic coefficients of all orders are arranged in order from low order to high; and arranging all Gaussian ellipsoids in the multi-stage sparse Gaussian ellipsoid set according to an interlaced data storage structure to generate an interlaced data sequence.
  6. 6. The method for lightweight encapsulation applied to a 3DGS model according to claim 5, wherein said parallel entropy encoder performs interleaving rearrangement and compression on the geometry attribute stream and the appearance attribute stream according to an access mode of the GPU buffer, and outputs a compressed code stream, and further comprising: Establishing independent first-order Markov models as context models of arithmetic coding for integer indexes of the geometric attribute stream respectively, and establishing static probability models based on adjacent data value statistical probabilities as context models of arithmetic coding for integer indexes of the appearance attribute stream; during encoding, according to the attribute category of the integer index to be encoded currently, selecting a corresponding context model to predict probability distribution of the integer index, and using an arithmetic encoder to perform compression encoding on the integer index according to the probability distribution to generate a compressed code stream.
  7. 7. The lightweight encapsulation method applied to the 3DGS model of claim 1, wherein said appending of multi-resolution hierarchical index and delta update metadata for the compressed bitstream comprises: Positioning compressed data segments belonging to each level sparse Gaussian ellipsoid set in the compressed code stream, and recording initial position offset and data segment length of the compressed data segments in the whole compressed code stream; Generating a hierarchy index table, and sequentially recording the initial position offset, the length of a data segment and a contribution score threshold value used when generating a corresponding hierarchy; For any two adjacent sparse Gaussian ellipsoid sets, identifying a new heightened Gaussian ellipsoid and an updated Gaussian ellipsoid which exists in two levels at the same time but has changed attribute values; the complete quantized attribute data of the new gauss ellipsoid and the attribute value differential data of the updated gauss ellipsoid changing between two levels are encoded together into an incremental updated data block; Generating incremental update metadata for each incremental update data block, wherein the incremental update metadata comprises a source hierarchy identification, a target hierarchy identification and storage position information of the incremental update data block in a package; the hierarchical index table and the delta update metadata are appended as header information before the compressed code stream.
  8. 8. The lightweight encapsulation method applied to the 3DGS model according to claim 7, wherein said packaging into a self-contained lightweight model encapsulation body in a network streaming loading order comprises: Taking the compressed code stream added with the hierarchical index and the incremental update metadata as an integral data body; adding a global file header at the forefront end of the whole data body, wherein the global file header comprises a file format identifier, a package version, a total package size and a total layer number; According to the sequence of network stream loading, the global file header, the hierarchy index table and the increment updating metadata of all increment updating data blocks are sequentially stored in the package, then the compressed code stream part corresponding to the lowest detail level is stored, and finally each increment updating data block and the corresponding compressed code stream part are sequentially stored according to the sequence from low to high detail level, so that the self-contained lightweight model package is formed.
  9. 9. The lightweight package method for 3DGS model according to claim 8, wherein said self-contained lightweight model package supports progressive decoding and rendering, specifically implemented as: downloading and analyzing a global file header and a hierarchical index table of the self-contained lightweight model package at a Web client; determining an initial target detail level according to the current network condition or the user requirement, and positioning and downloading a compressed code stream segment corresponding to the target detail level according to the level index table; Inputting the downloaded compressed code stream segment into a decoder corresponding to an arithmetic encoder, performing arithmetic decoding by using the same context model as that used in encoding, restoring an integer index data sequence which corresponds to a target detail level and is staggered, and transmitting the restored integer index data sequence to a WebGPU rendering pipeline; In the shader of WebGPU, converting the integer index data back to the floating point type geometrical attribute and the appearance attribute of the rendering in real time by utilizing inverse quantization parameters consistent with the coding end, and using the integer index data for rendering; when the rendering detail is promoted, the client locates and downloads the increment updating data block from the current level to the target detail level according to the increment updating metadata, and the increment updating data block is combined with the existing data after decoding to obtain a sparse Gaussian ellipsoid set of the target detail level for rendering.
  10. 10. A lightweight packaging system applied to a 3DGS model for implementing the lightweight packaging method applied to a 3DGS model according to any one of claims 1 to 9, comprising: The data receiving and quantizing module is used for receiving the Gaussian ellipsoid point cloud data stream generated by the 3D Gaussian splatter modeling, decoupling the Gaussian ellipsoid point cloud data stream into a geometric attribute stream and an appearance attribute stream, and respectively carrying out self-adaptive precision quantization encoding on the geometric attribute stream and the appearance attribute stream to generate quantized attribute data blocks; The hierarchical screening module is used for carrying out importance sorting and hierarchical screening on the contribution degree of the final rendered image according to each ellipsoid by taking the quantized attribute data block as input to generate a multi-level sparse Gaussian ellipsoid set; The parallel coding module is used for inputting the multi-stage sparse Gaussian ellipsoid set into a parallel entropy coder, and the parallel entropy coder performs interleaving rearrangement and compression on the geometric attribute stream and the appearance attribute stream according to the access mode of the GPU buffer region and outputs a compressed code stream; And the packaging module is used for adding multi-resolution level indexes and increment updating metadata for the compressed code stream and packaging the compressed code stream into a self-contained lightweight model packaging body according to the network stream loading sequence.

Description

Lightweight packaging method and system applied to 3DGS model Technical Field The invention belongs to the technical field of 3D modeling and packaging, and particularly relates to a lightweight packaging method and system applied to a 3DGS model. Background The 3D Gaussian splatter modeling becomes a mainstream technology of three-dimensional scene reconstruction due to high rendering precision and efficiency, but the generated Gaussian ellipsoid point cloud data contains massive geometric and appearance attribute information, so that the volume of the data is huge, and the real-time streaming and Web end rendering requirements in a network environment are difficult to be directly adapted. The existing 3DGS model packaging method mostly adopts a general data compression strategy, is not optimized for the attribute characteristics of Gaussian ellipsoids and the hardware access characteristics of WebGPU, has the problems of fixed quantization precision and mismatching of data sequencing and GPU memory layout, and causes difficulty in considering compression rate and decoding efficiency. Meanwhile, the traditional method lacks of differentiated processing of Gaussian ellipsoid rendering contribution degree, does not construct a multi-level sparse data set, cannot realize progressive rendering of a model, and is easy to solve the problems of rendering blocking or incoherent detail loading when the network bandwidth fluctuates. In addition, the increment updating capability of the existing packaging scheme is insufficient, the complete data is needed to be reloaded when the hierarchy is switched, the network transmission overhead is greatly increased, the decoding process is mostly executed in series, the parallel computing characteristic of the Web end is not matched, and the utilization rate of a rendering pipeline of WebGPU is further reduced. Therefore, a light-weight packaging method adapting to the 3DGS model features and Web end deployment requirements is needed, and the self-adaptive quantization of attributes, GPU-friendly compression coding and multi-resolution hierarchical packaging are realized, so that the network transmission efficiency and the progressive rendering performance of the Web end of the model are improved on the premise of ensuring the rendering effect. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a lightweight packaging method and a lightweight packaging system applied to a 3DGS model, wherein Gaussian ellipsoid point cloud data streams of 3D Gaussian splatter modeling are decoupled into geometric and appearance attribute streams, attribute data blocks are generated through self-adaptive precision quantization coding, a multi-level sparse Gaussian ellipsoid set is generated by sorting and screening contribution degrees of ellipsoids to a rendering image, the attribute streams are staggered and rearranged through a parallel entropy coder and compressed to obtain compressed code streams, multi-resolution hierarchical indexes and incremental update metadata are added to the compressed code streams, and the compressed code streams are packaged into self-contained lightweight model packages according to network flow loading sequence. In order to achieve the above purpose, the present invention provides the following technical solutions: the lightweight packaging method applied to the 3DGS model comprises the following steps: Receiving a Gaussian ellipsoid point cloud data stream generated by 3D Gaussian splats modeling, decoupling the Gaussian ellipsoid point cloud data stream into a geometric attribute stream and an appearance attribute stream, and respectively carrying out self-adaptive precision quantization coding on the geometric attribute stream and the appearance attribute stream to generate quantized attribute data blocks; taking the quantized attribute data blocks as input, and carrying out importance sorting and hierarchical screening on the contribution degree of the final rendered image according to each ellipsoid to generate a multi-level sparse Gaussian ellipsoid set, wherein each level of data set is a strict subset of the upper level; Inputting the multi-level sparse Gaussian ellipsoid set into a parallel entropy coder specially optimized for WebGPU memory layout, and performing interleaving rearrangement and compression on the geometric attribute stream and the appearance attribute stream by the parallel entropy coder according to the access mode of the GPU buffer region to output a compressed code stream; And adding multi-resolution level indexes and increment updating metadata for the compressed code stream, and packaging the compressed code stream into a self-contained lightweight model package according to the network stream loading sequence. Specifically, the receiving a gaussian ellipsoid point cloud data stream generated by 3D gaussian splatter modeling, and decoupling the gaussian ellipsoid point cloud data stream into a g