KR-102963116-B1 - METHOD AND APPARATUS FOR COMPRESSING 3D GAUSSIAN SPLATTING DATA USING SINGULAR VALUE DECOMPOSITION

KR102963116B1KR 102963116 B1KR102963116 B1KR 102963116B1KR-102963116-B1

Abstract

According to one aspect of the present disclosure, a method for compressing 3D Gaussian Splatting (3DGS) data using Singular Value Decomposition (SVD), which is performed by a computing device, is disclosed. The method comprises: generating a plurality of voxels by hierarchically voxelizing raw 3DGS data including a plurality of 3D Gaussian ellipsoids; generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids, wherein the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes; and producing a plurality of singular value decomposition matrices for each of the at least one local Gaussian graph by performing singular value decomposition on a plurality of spherical harmonic coefficients corresponding to each of the at least one local Gaussian graph. and may include the step of generating compressed 3DGS data by performing a low-rank approximation on the plurality of singular value decomposition matrices.

Inventors

카나니 쿨와-주마 카나니

Assignees

(주)딥인사이트

Dates

Publication Date: 20260511
Application Date: 20251103

Claims (19)

A method for compressing 3D Gaussian Splatting (3DGS) data using Singular Value Decomposition (SVD) performed by a computing device, A step of generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; A step of generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids—the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes—; A step of producing a plurality of singular value decomposition matrices for each of the at least one local Gaussian graph by performing singular value decomposition on a plurality of spherical harmonic coefficients corresponding to each of the at least one local Gaussian graph; and A step of generating compressed 3DGS data by performing a low-rank approximation on the plurality of singular value decomposition matrices; Includes, The multiple first singular value decomposition matrices included in the first local Gaussian graph are: A plurality of first luminance singular value decomposition matrices corresponding to the luminance channels of the YUV color space; A plurality of first blue color difference singular value decomposition matrices corresponding to the blue color difference channel of the above YUV color space; and A plurality of first red color difference singular value decomposition matrices corresponding to the red color difference channel of the above YUV color space; Includes, The step of generating the above compressed 3DGS data is: A step of approximating the plurality of first luminance singular value decomposition matrices by identifying upper luminance singular values and upper luminance singular vectors of a preset first rank number in the plurality of first luminance singular value decomposition matrices; A step of approximating the plurality of first blue color difference singular value decomposition matrices by identifying upper blue color difference singular values and upper blue color difference singular vectors of a preset second rank number in the plurality of first blue color difference singular value decomposition matrices; A step of approximating the plurality of first red color difference singular value decomposition matrices by identifying upper red color difference singular values and upper red color difference singular vectors of the preset second rank number in the plurality of first red color difference singular value decomposition matrices; and A step of generating the compressed 3DGS data including the approximated plurality of first luminance singular value decomposition matrices, the approximated plurality of first blue chrominance singular value decomposition matrices, and the approximated plurality of first red chrominance singular value decomposition matrices; Includes, and The above-mentioned preset first rank number has a value greater than the above-mentioned preset second rank number, method.
In Article 1, Each of the above plurality of 3D Gaussian ellipsoids has a set of parameters including a center position, a normal vector, a scale vector, a rotation quaternion, opacity, and color parameters, and The above color parameter includes the plurality of spherical harmonic coefficients for each of the plurality of color channels, method.
In Article 2, The above plurality of spherical harmonic coefficients are: A single view-independent spherical harmonic coefficient representing a color component independent of viewpoint change; and It includes multiple view-dependent spherical harmonic coefficients representing color components dependent on changes in viewpoint, and The above time-independent spherical harmonic coefficients correspond to a spherical harmonic function with a higher spherical harmonic order of 0, and, Each of the above plurality of time-dependent spherical harmonic coefficients corresponds to a spherical harmonic function having a higher spherical harmonic order of 1 or greater, method.
In Article 2, The above plurality of spherical harmonic coefficients are defined based on spherical harmonic order pairs including an upper spherical harmonic order and a lower spherical harmonic order, and The number of the above plurality of spherical harmonic coefficients is determined based on the maximum value of a preset upper spherical harmonic order, and The range of the lower spherical harmonic order included in the first upper spherical harmonic order is determined based on the first upper spherical harmonic order, method.
In Article 1, The method further comprises the step of reducing the data size of the raw 3DGS data by removing from the raw 3DGS data at least one 3D Gaussian ellipsoid among the plurality of 3D Gaussian ellipsoids, the opacity of which is less than or equal to a preset threshold opacity. method.
In Article 1, The plurality of voxels are adaptively generated based on the density of the plurality of 3D Gaussian ellipsoids, and The size of each of the plurality of voxels has a negative correlation with the density of the plurality of three-dimensional Gaussian ellipsoids, method.
In Article 1, The step of generating at least one local Gaussian graph above is, The method includes the step of generating at least one local Gaussian graph that defines the connectivity of the K 3D Gaussian ellipsoids closest to each of the plurality of voxels by applying a K-NN (K-Nearest Neighbor) algorithm to the 3D Gaussian ellipsoids included in each of the plurality of voxels, wherein K is a natural number, and Two 3D Gaussian ellipsoids connected by an edge, wherein the distance between their center positions within at least one local Gaussian graph is less than or equal to a preset threshold distance, method.
In Article 1, Prior to the step of calculating the plurality of singular value decomposition matrices mentioned above, The method further includes the step of converting a plurality of spherical harmonic coefficients defined in the RGB color space for each of the three-dimensional Gaussian ellipsoids included in the at least one local Gaussian graph into a plurality of spherical harmonic coefficients defined in the YUV color space using a conversion formula between the RGB color space and the YUV color space, The above RGB color space includes a red channel, a green channel, and a blue channel, and The above YUV color space includes a luminance channel, a blue chrominance channel, and a red chrominance channel, method.
In Article 1, The step of calculating the above plurality of singular value decomposition matrices is: A step of calculating a first spherical harmonic coefficient matrix for each color channel of a first local Gaussian graph; and A step of calculating a plurality of first singular value decomposition matrices for each color channel of the first local Gaussian graph by performing singular value decomposition on the first spherical harmonic coefficient matrix for each color channel; Includes, and The first spherical harmonic coefficient matrix corresponding to the first color channel corresponds to a matrix having each of the three-dimensional Gaussian ellipsoids included in the first local Gaussian graph as rows and each of the plurality of first spherical harmonic coefficients belonging to the first color channel of each three-dimensional Gaussian ellipsoid as column values, method.
In Article 9, The above multiple first singular value decomposition matrices for each color channel are: A left singular vector matrix corresponding to the first spherical harmonic coefficient matrix for each color channel; A singular value matrix corresponding to the first spherical harmonic coefficient matrix for each color channel; and A right singular vector matrix corresponding to the first spherical harmonic coefficient matrix for each color channel; including, method.
delete
In Article 1, The above upper luminance singular values correspond to the above preset number of first rank luminance singular values identified based on the order of largest values among the luminance singular values included in the plurality of first luminance singular value decomposition matrices, and each of the above upper luminance singular vectors corresponds to each of the above upper luminance singular values, and The above upper blue color difference singularities correspond to the blue color difference singularities of the above preset second rank number identified based on the order of magnitude among the blue color difference singularities included in the plurality of first blue color difference singularity decomposition matrices, and each of the above upper blue color difference singularity vectors corresponds to each of the above upper blue color difference singularities, and The above upper red color difference singularities correspond to the above preset second rank number of red color difference singularities identified based on the order of magnitude among the red color difference singularities included in the plurality of first red color difference singularity decomposition matrices, and each of the above upper red color difference singularity vectors corresponds to each of the above upper red color difference singularities, method.
In Article 1, The step of generating the above compressed 3DGS data is: A step of quantizing a first luminance singular value matrix included in the aforementioned approximated plurality of first luminance singular value decomposition matrices with a preset first quantization precision to produce a quantized first luminance singular value matrix; A step of quantizing a first blue color difference singular value matrix included in the aforementioned approximated plurality of first blue color difference singular value decomposition matrices with a preset second quantization precision to produce a quantized first blue color difference singular value matrix; and A step of quantizing a first red color difference singular value matrix included in the aforementioned approximated plurality of first red color difference singular value decomposition matrices with the aforementioned preset second quantization precision to produce a quantized first red color difference singular value matrix; A step of generating the compressed 3DGS data including the quantized first luminance singular value matrix, the quantized first blue chrominance singular value matrix, and the quantized first red chrominance singular value matrix; Includes, and The above-mentioned preset first quantization precision has a higher precision than the above-mentioned preset second quantization precision, method.
In Article 1, The step of generating the above compressed 3DGS data is: A step of generating a plurality of first luminance entropy codes by applying a first luminance singular value matrix included in the plurality of approximated first luminance singular value decomposition matrices above; A step of generating a plurality of first blue color difference entropy codes by applying a second entropy encoding function to a first blue color difference singular value matrix included in the plurality of approximated first blue color difference singular value decomposition matrices; A step of generating a plurality of first red color difference entropy codes by applying a second entropy encoding function to a first red color difference singular value matrix included in the plurality of approximated first red color difference singular value decomposition matrices; and A step of generating the compressed 3DGS data including the plurality of first luminance entropy codes, the plurality of first blue color difference entropy codes, and the plurality of first red color difference entropy codes; including, method.
In Article 1, After the step of generating the above compressed 3DGS data, A step of calculating a plurality of reconstructed spherical harmonic coefficients of 3D Gaussian ellipsoids included in each of the at least one local Gaussian graph by performing singular value decomposition reconstruction (SVD reconstruction) on a plurality of singular value decomposition matrices included in the compressed 3DGS data; and A step of generating reconstructed 3DGS data based on the above plurality of reconstructed spherical harmonic coefficients; including, method.
In Article 15, Prior to the step of calculating the above plurality of restored spherical harmonic coefficients, A step of entropy decoding a plurality of entropy codes included in the above compressed 3DGS data; or A step of dequantizing the quantized singular value decomposition matrices included in the above compressed 3DGS data; including at least one more of, method.
In Article 15, The step of generating the above-mentioned restored 3DGS data is: A step comprising converting a plurality of reconstructed spherical harmonic coefficients defined in the YUV color space of each of the plurality of 3D Gaussian ellipsoids included in the reconstructed 3DGS data into a plurality of reconstructed spherical harmonic coefficients defined in the RGB color space using a conversion formula between the YUV color space and the RGB color space, method.
A computing device that compresses 3D Gaussian Splatting (3DGS) data using Singular Value Decomposition (SVD), It includes a processor comprising at least one core, and The above processor is, Generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; Based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids, at least one local Gaussian graph is generated within each of the plurality of voxels - the local Gaussian graph consists of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes -; By performing singular value decomposition on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graphs, a plurality of singular value decomposition matrices for each of the at least one local Gaussian graphs are produced; and Compressed 3DGS data is generated by performing a low-rank approximation on the plurality of singular value decomposition matrices mentioned above, and The multiple first singular value decomposition matrices included in the first local Gaussian graph are: A plurality of first luminance singular value decomposition matrices corresponding to the luminance channels of the YUV color space; A plurality of first blue color difference singular value decomposition matrices corresponding to the blue color difference channel of the above YUV color space; and A plurality of first red color difference singular value decomposition matrices corresponding to the red color difference channel of the above YUV color space; Includes, The above processor is, In generating the above compressed 3DGS data: By identifying upper luminance singular values and upper luminance singular vectors of a preset first rank number in the plurality of first luminance singular value decomposition matrices, the plurality of first luminance singular value decomposition matrices are approximated; By identifying upper blue color difference singular values and upper blue color difference singular vectors of a preset second rank number in the plurality of first blue color difference singular value decomposition matrices, the plurality of first blue color difference singular value decomposition matrices are approximated; By identifying upper red color difference singular values and upper red color difference singular vectors of the preset second rank number in the plurality of first red color difference singular value decomposition matrices, the plurality of first red color difference singular value decomposition matrices are approximated; and Generating the compressed 3DGS data including the approximated plurality of first luminance singular value decomposition matrices, the approximated plurality of first blue chrominance singular value decomposition matrices, and the approximated plurality of first red chrominance singular value decomposition matrices, and The above-mentioned preset first rank number has a value greater than the above-mentioned preset second rank number, Computing device.
A computer program stored on a computer-readable storage medium, wherein the computer program causes a processor of a computing device to perform a method for compressing 3D Gaussian Splatting (3DGS) data using Singular Value Decomposition (SVD), and the method comprises: A step of generating multiple voxels by hierarchically voxelizing raw 3DGS data containing multiple 3D Gaussian ellipsoids; A step of generating at least one local Gaussian graph within each of the plurality of voxels based on the distance between the center positions of each of the plurality of 3D Gaussian ellipsoids—the local Gaussian graph is composed of nodes corresponding to the 3D Gaussian ellipsoids and edges connecting the two nodes—; A step of producing a plurality of singular value decomposition matrices for each of the at least one local Gaussian graph by performing singular value decomposition on a plurality of spherical harmonic coefficients included in each of the at least one local Gaussian graphs; and A step of generating compressed 3DGS data by performing a low-rank approximation on the plurality of singular value decomposition matrices; Includes, The multiple first singular value decomposition matrices included in the first local Gaussian graph are: A plurality of first luminance singular value decomposition matrices corresponding to the luminance channels of the YUV color space; A plurality of first blue color difference singular value decomposition matrices corresponding to the blue color difference channel of the above YUV color space; and A plurality of first red color difference singular value decomposition matrices corresponding to the red color difference channel of the above YUV color space; Includes, The step of generating the above compressed 3DGS data is: A step of approximating the plurality of first luminance singular value decomposition matrices by identifying upper luminance singular values and upper luminance singular vectors of a preset first rank number in the plurality of first luminance singular value decomposition matrices; A step of approximating the plurality of first blue color difference singular value decomposition matrices by identifying upper blue color difference singular values and upper blue color difference singular vectors of a preset second rank number in the plurality of first blue color difference singular value decomposition matrices; A step of approximating the plurality of first red color difference singular value decomposition matrices by identifying upper red color difference singular values and upper red color difference singular vectors of the preset second rank number in the plurality of first red color difference singular value decomposition matrices; and A step of generating the compressed 3DGS data including the approximated plurality of first luminance singular value decomposition matrices, the approximated plurality of first blue chrominance singular value decomposition matrices, and the approximated plurality of first red chrominance singular value decomposition matrices; Includes, and The above-mentioned preset first rank number has a value greater than the above-mentioned preset second rank number, A computer program stored on a computer-readable storage medium.

Description

Method and apparatus for compressing 3D Gaussian splatting data using singular value decomposition The present disclosure relates to 3D scene visualization and reconstruction technology, and specifically to a method and apparatus for compressing three-dimensional Gaussian splatting data using singular value decomposition. 3D scene reconstruction technology plays a key role in various fields such as autonomous driving, augmented reality, virtual reality, 3D scanning, video editing, and visual effects industries, and improving real-time or high-resolution 3D modeling and rendering performance is emerging as an important research and industrial topic. The field of 3D scene reconstruction may include technologies for extracting and restoring the 3D shape and structure of the real world from 2D images. This field is a convergence of computer vision and computer graphics, aiming to precisely reconstruct objects and scenes within 3D space based on 2D data captured from multiple viewpoints. In the field of 3D scene reconstruction, methods applying efficient data processing and optimization algorithms have been proposed to display 2D data in a 3D form in real time. In particular, 3D Gaussian Splatting (3DGS) is gaining attention as a 3D scene reconstruction technique for achieving both real-time rendering speed and high visual quality. 3DGS allows a 3D scene to be explicitly represented as a set of 3D Gaussian ellipsoids instead of a complex neural network. Each of these Gaussians can possess attributes such as position, size, opacity, and color. The 3DGS technique can be initialized and optimized using images of consecutive frames and Sparse Point Cloud data obtained through Structure-from-Motion (SfM). Such methods can be utilized in the surrounding environment perception of autonomous vehicles, real-time spatial perception in Augmented Reality (AR) and Virtual Reality (VR), 3D scanning and digital twin technology, real-time video editing, and the generation of visual effects. Various aspects are now described with reference to the drawings, wherein similar reference numbers are used to collectively refer to similar components. In the following embodiments, for illustrative purposes, a number of specific details are presented to provide a comprehensive understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details. In other examples, known structures and devices are illustrated in block diagram form to facilitate the description of one or more aspects. FIG. 1 is a block diagram of a computing device that compresses three-dimensional Gaussian splatting data using singular value decomposition according to one embodiment of the present disclosure. FIG. 2 is a flowchart illustrating a method for a computing device according to one embodiment of the present disclosure to compress three-dimensional Gaussian splatting data using singular value decomposition. FIG. 3 is a diagram illustrating a plurality of voxels generated from three-dimensional Gaussian splatting data through hierarchical voxelization according to one embodiment of the present disclosure. FIG. 4 is a drawing showing a local Gaussian graph including three-dimensional Gaussian ellipsoids according to one embodiment of the present disclosure. FIG. 5 is a flowchart illustrating a method for a computing device according to one embodiment of the present disclosure to compress three-dimensional Gaussian splatting data using various compression techniques. FIG. 6 is a flowchart illustrating a method for a computing device according to one embodiment of the present disclosure to restore 3D Gaussian splatting data from compressed 3D Gaussian splatting data. FIG. 7 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented. Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions. As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and/or a computer. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and/or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may