Search

KR-102963115-B1 - METHOD AND APPARATUS FOR COMPRESSING 3D GAUSSIAN SPLATTING DATA USING PRINCIPAL COMPONENT ANALYSIS

KR102963115B1KR 102963115 B1KR102963115 B1KR 102963115B1KR-102963115-B1

Abstract

According to one aspect of the present disclosure, a method for compressing 3D Gaussian Splatting (3DGS) data using Principal Component Analysis (PCA), performed by a computing device, is disclosed. The method comprises: generating a plurality of voxels by hierarchically voxelizing raw 3DGS data including a plurality of 3D Gaussian ellipsoids; generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids, wherein the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes; and calculating a principal component score matrix for each of the at least one local Gaussian graph by performing principal component analysis on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graph. and may include the step of generating compressed 3DGS data based on the principal component score matrix above.

Inventors

  • 카나니 쿨와-주마 카나니

Assignees

  • (주)딥인사이트

Dates

Publication Date
20260511
Application Date
20251028

Claims (19)

  1. A method for compressing 3D Gaussian Splatting (3DGS) data using Principal Component Analysis (PCA) performed by a computing device, A step of generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; A step of generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids—the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes—; A step of calculating a principal component score matrix for each of the at least one local Gaussian graph by performing principal component analysis on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graphs; and A step of generating compressed 3DGS data based on the above principal component score matrix; Includes, Each of the above at least one local Gaussian graph includes a local Gaussian subgraph corresponding to each of the plurality of color channels, and A plurality of local Gaussian subgraphs included in each of the above at least one local Gaussian graph have the same topology, and The step of calculating the principal component score matrix for each of the above at least one local Gaussian graph is: A step of calculating a first spherical harmonic coefficient matrix for a first local Gaussian subgraph corresponding to a first color channel—the first spherical harmonic coefficient matrix has as components first spherical harmonic coefficients corresponding to the first color channel of the 3D Gaussian ellipsoids included in the first local Gaussian subgraph—; A step of calculating a first low-rank matrix for the first spherical harmonic coefficient matrix by performing Robust Principal Component Analysis (RPCA) on the first spherical harmonic coefficient matrix; and A step of calculating a first principal component score matrix for the first local Gaussian subgraph by performing principal component analysis on the first low-rank matrix; Includes, and The first spherical harmonic coefficient matrix corresponds to a matrix in which each of the three-dimensional Gaussian ellipsoids included in the first local Gaussian subgraph is in the rows and each of the plurality of first spherical harmonic coefficients of each three-dimensional Gaussian ellipsoid is in the columns. method.
  2. In Article 1, Each of the above plurality of 3D Gaussian ellipsoids has a set of parameters including a center position, a normal vector, a scale vector, a rotation quaternion, opacity, and color parameters, and The above color parameter includes the plurality of spherical harmony coefficients for each of the plurality of color channels, method.
  3. In Article 2, The above plurality of spherical harmonic coefficients are: A single view-independent spherical harmonic coefficient representing a color component independent of viewpoint change; and It includes multiple view-dependent spherical harmonic coefficients representing color components dependent on changes in viewpoint, and The above time-independent spherical harmonic coefficients correspond to a spherical harmonic function with a higher spherical harmonic order of 0, and, Each of the above plurality of time-dependent spherical harmonic coefficients corresponds to a spherical harmonic function having a higher spherical harmonic order of 1 or greater, method.
  4. In Article 2, The above plurality of spherical harmonic coefficients are defined based on spherical harmonic order pairs including an upper spherical harmonic order and a lower spherical harmonic order, and The number of the above plurality of spherical harmonic coefficients is determined based on the maximum value of a preset upper spherical harmonic order, and The range of the lower spherical harmonic order included in the first upper spherical harmonic order is determined based on the first upper spherical harmonic order, method.
  5. In Article 1, The plurality of voxels are adaptively generated based on the density of the plurality of 3D Gaussian ellipsoids, and The size of each of the plurality of voxels has a negative correlation with the density of the plurality of three-dimensional Gaussian ellipsoids, method.
  6. In Article 1, The step of generating at least one local Gaussian graph above is, The method includes the step of generating at least one local Gaussian graph that defines the connectivity of the K 3D Gaussian ellipsoids closest to each of the plurality of voxels by applying a K-NN (K-Nearest Neighbor) algorithm to the 3D Gaussian ellipsoids included in each of the plurality of voxels, wherein K is a natural number, and Two 3D Gaussian ellipsoids connected by an edge, wherein the distance between their center positions within at least one local Gaussian graph is less than or equal to a preset threshold distance, method.
  7. delete
  8. delete
  9. delete
  10. A method for compressing 3D Gaussian Splatting (3DGS) data using Principal Component Analysis (PCA) performed by a computing device, A step of generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; A step of generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids—the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes—; A step of calculating a principal component score matrix for each of the at least one local Gaussian graph by performing principal component analysis on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graphs; and A step of generating compressed 3DGS data based on the above principal component score matrix; Includes, Each of the above at least one local Gaussian graph includes a local Gaussian subgraph corresponding to each of the plurality of color channels, and A plurality of local Gaussian subgraphs included in each of the above at least one local Gaussian graph have the same topology, and The step of calculating the principal component score matrix for each of the above at least one local Gaussian graph is: A step of calculating a first spherical harmonic coefficient matrix for a first local Gaussian subgraph corresponding to a first color channel—the first spherical harmonic coefficient matrix has as components first spherical harmonic coefficients corresponding to the first color channel of the 3D Gaussian ellipsoids included in the first local Gaussian subgraph—; A step of calculating a first low-rank matrix for the first spherical harmonic coefficient matrix by performing Robust Principal Component Analysis (RPCA) on the first spherical harmonic coefficient matrix; and A step of calculating a first principal component score matrix for the first local Gaussian subgraph by performing principal component analysis on the first low-rank matrix; Includes, and The step of calculating the first principal component score matrix for the first local Gaussian subgraph above is: A step of calculating a first covariance matrix for the first low-rank matrix; A step of calculating the explained variance ratio of each principal component included in the first local Gaussian subgraph by performing eigen-decomposition on the first covariance matrix; and A step of calculating the first principal component score matrix based on the above-described variance ratio; including, method.
  11. In Article 10, The first principal component score matrix is composed of the principal component scores of the minimum number of principal components such that the cumulative explained variance ratio is greater than or equal to a preset first threshold explained variance ratio, based on the order of the largest explained variance ratios. method.
  12. In Article 10, The first principal component score matrix above is calculated by excluding principal components whose explained variance ratio is less than or equal to a preset second threshold explained variance ratio, method.
  13. In Article 1, The step of generating the above compressed 3DGS data is: A step of performing quantization on a first principal component score matrix of a first local Gaussian subgraph corresponding to a first color channel to produce a quantized first principal component score matrix; and A step of generating the compressed 3DGS data including the quantized first principal component score matrix; including, method.
  14. In Article 1, The step of generating the above compressed 3DGS data is: A step of generating a plurality of entropy codes by applying an entropy encoding function to the principal component scores included in the first principal component score matrix of the first local Gaussian subgraph corresponding to the first color channel; and A step of generating the compressed 3DGS data including the plurality of entropy codes; including, method.
  15. In Article 1, After the step of generating the above compressed 3DGS data, A step of calculating a plurality of reconstructed spherical harmonic coefficients included in each of the at least one local Gaussian graph by performing an inverse PCA on the principal component score matrices included in the compressed 3DGS data; and A step of generating reconstructed 3DGS data based on the above plurality of reconstructed spherical harmonic coefficients; including, method.
  16. In Article 15, Prior to the step of calculating the above plurality of restored spherical harmonic coefficients, A step of entropy decoding a plurality of entropy codes included in the above compressed 3DGS data—wherein the entropy codes correspond to entropy-encoded values of principal component scores—; or A step of dequantizing the quantized principal component scores included in the above compressed 3DGS data; including at least one more of, method.
  17. In Article 15, The step of calculating the above plurality of restored spherical harmonic coefficients is: A step of calculating a reconstructed low-rank matrix corresponding to each of the at least one local Gaussian graph by performing an inverse principal component transform on the principal component score matrices included in the compressed 3DGS data; and A step of calculating the plurality of reconstructed spherical harmonic coefficients included in each of the at least one local Gaussian graph by performing a robust inverse principal component transformation (Inverse RPCA) on the reconstructed low-rank matrix; including, method.
  18. A computing device that compresses 3D Gaussian Splatting (3DGS) data using Principal Component Analysis (PCA), It includes a processor comprising at least one core, and The above processor is, Generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; Based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids, at least one local Gaussian graph is generated within each of the plurality of voxels - the local Gaussian graph consists of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes -; By performing principal component analysis on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graphs, a principal component score matrix for each of the at least one local Gaussian graphs is calculated; and Based on the above principal component score matrix, compressed 3DGS data is generated, and Each of the above at least one local Gaussian graph includes a local Gaussian subgraph corresponding to each of the plurality of color channels, and A plurality of local Gaussian subgraphs included in each of the above at least one local Gaussian graph have the same topology, and The above processor is, In calculating the principal component score matrix for each of the above at least one local Gaussian graph: Calculate a first spherical harmonic coefficient matrix for a first local Gaussian subgraph corresponding to a first color channel, wherein the first spherical harmonic coefficient matrix has as components the first spherical harmonic coefficients corresponding to the first color channel of the 3D Gaussian ellipsoids included in the first local Gaussian subgraph. By performing Robust Principal Component Analysis (RPCA) on the first spherical harmonic coefficient matrix, a first low-rank matrix for the first spherical harmonic coefficient matrix is calculated; and By performing principal component analysis on the first low-rank matrix above, a first principal component score matrix for the first local Gaussian subgraph is calculated, and The first spherical harmonic coefficient matrix corresponds to a matrix in which each of the three-dimensional Gaussian ellipsoids included in the first local Gaussian subgraph is in the rows and each of the plurality of first spherical harmonic coefficients of each three-dimensional Gaussian ellipsoid is in the columns. Computing device.
  19. A computer program stored on a computer-readable storage medium, wherein the computer program causes a processor of a computing device to perform a method for compressing 3D Gaussian Splatting (3DGS) data using Principal Component Analysis (PCA), and the method comprises: A step of generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; A step of generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids—the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes—; A step of calculating a principal component score matrix for each of the at least one local Gaussian graph by performing principal component analysis on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graphs; and A step of generating compressed 3DGS data based on the above principal component score matrix; Includes, Each of the above at least one local Gaussian graph includes a local Gaussian subgraph corresponding to each of the plurality of color channels, and A plurality of local Gaussian subgraphs included in each of the above at least one local Gaussian graph have the same topology, and The step of calculating the principal component score matrix for each of the above at least one local Gaussian graph is: A step of calculating a first spherical harmonic coefficient matrix for a first local Gaussian subgraph corresponding to a first color channel—the first spherical harmonic coefficient matrix has as components first spherical harmonic coefficients corresponding to the first color channel of the 3D Gaussian ellipsoids included in the first local Gaussian subgraph—; A step of calculating a first low-rank matrix for the first spherical harmonic coefficient matrix by performing Robust Principal Component Analysis (RPCA) on the first spherical harmonic coefficient matrix; and A step of calculating a first principal component score matrix for the first local Gaussian subgraph by performing principal component analysis on the first low-rank matrix; Includes, and The first spherical harmonic coefficient matrix corresponds to a matrix in which each of the three-dimensional Gaussian ellipsoids included in the first local Gaussian subgraph is in the rows and each of the plurality of first spherical harmonic coefficients of each three-dimensional Gaussian ellipsoid is in the columns. A computer program stored on a computer-readable storage medium.

Description

Method and apparatus for compressing 3D Gaussian splatting data using principal component analysis The present disclosure relates to 3D scene visualization and reconstruction technology, and specifically to a method and apparatus for compressing three-dimensional Gaussian splatting data using principal component analysis. 3D scene reconstruction technology plays a key role in various fields such as autonomous driving, augmented reality, virtual reality, 3D scanning, video editing, and visual effects industries, and improving real-time or high-resolution 3D modeling and rendering performance is emerging as an important research and industrial topic. The field of 3D scene reconstruction may include technologies for extracting and restoring the 3D shape and structure of the real world from 2D images. This field is a convergence of computer vision and computer graphics, aiming to precisely reconstruct objects and scenes within 3D space based on 2D data captured from multiple viewpoints. In the field of 3D scene reconstruction, methods applying efficient data processing and optimization algorithms have been proposed to display 2D data in a 3D form in real time. In particular, 3D Gaussian Splatting (3DGS) is gaining attention as a 3D scene reconstruction technique for achieving both real-time rendering speed and high visual quality. 3DGS allows a 3D scene to be explicitly represented as a set of 3D Gaussian ellipsoids instead of a complex neural network. Each of these Gaussians can possess attributes such as position, size, opacity, and color. The 3DGS technique can be initialized and optimized using images of consecutive frames and Sparse Point Cloud data obtained through Structure-from-Motion (SfM). Such methods can be utilized in the surrounding environment perception of autonomous vehicles, real-time spatial perception in Augmented Reality (AR) and Virtual Reality (VR), 3D scanning and digital twin technology, real-time video editing, and the generation of visual effects. Various aspects are now described with reference to the drawings, wherein similar reference numbers are used to collectively refer to similar components. In the following embodiments, for illustrative purposes, a number of specific details are presented to provide a comprehensive understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details. In other examples, known structures and devices are illustrated in block diagram form to facilitate the description of one or more aspects. FIG. 1 is a block diagram of a computing device that compresses three-dimensional Gaussian splatting data using principal component analysis according to one embodiment of the present disclosure. FIG. 2 is a flowchart illustrating a method for a computing device according to one embodiment of the present disclosure to compress three-dimensional Gaussian splatting data using principal component analysis. FIG. 3 is a diagram illustrating a plurality of voxels generated from three-dimensional Gaussian splatting data through hierarchical voxelization according to one embodiment of the present disclosure. FIG. 4 is a diagram showing a local Gaussian subgraph by color channel according to one embodiment of the present disclosure. FIG. 5 is a diagram illustrating a principal component analysis graph for spherical harmonic coefficients according to one embodiment of the present disclosure. FIG. 6 is a flowchart illustrating a method for a computing device according to one embodiment of the present disclosure to compress three-dimensional Gaussian splatting data using various compression techniques. FIG. 7 is a flowchart illustrating a method for a computing device according to one embodiment of the present disclosure to restore 3D Gaussian splatting data from compressed 3D Gaussian splatting data. FIG. 8 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented. Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions. As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and/or a computer. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and/or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally,