CN-121982183-A - Gaussian splatter covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm

CN121982183ACN 121982183 ACN121982183 ACN 121982183ACN-121982183-A

Abstract

The invention discloses a Gaussian splatter covariance matrix optimization method based on a Hamiltonian Monte Carlo algorithm, and belongs to the technical field of 3D Gaussian splatter modeling. The invention builds a dynamics system through a Hamiltonian Monte Carlo algorithm, further builds a dynamics equation to improve the processing capacity and processing precision of a covariance matrix in optimizing 3D Gaussian splatter on Gaussian ellipsoid parameters, realizes and obtains a brand-new covariance matrix after optimization through a quality matrix by building Hamiltonian amount, introducing momentum variables, simulating dynamics through a frog-leaping method and the like, provides an optimization and analysis method of the Gaussian splatter covariance matrix based on the Hamiltonian Monte Carlo algorithm, and simultaneously provides a peak signal to noise ratio (PSNR) evaluation mechanism for quantizing and optimizing results.

Inventors

SHI GUIGANG
WANG HAN
LI YINGQI
ZHU HUIQIU
ZUO GUANGZHI

Assignees

安徽建筑大学

Dates

Publication Date: 20260505
Application Date: 20260408

Claims (10)

1. The Gaussian splatter covariance matrix optimization method based on the Hamiltonian Monte Carlo algorithm is characterized by comprising the following steps of: S1, data acquisition and preprocessing Building image data are collected through the unmanned aerial vehicle, and are preprocessed, so that the building image data with high resolution are obtained; s2. covariance matrix construction Constructing a covariance matrix for describing the shape and direction of the Gaussian ellipsoid; s3, introducing momentum variable Constructing Hamiltonian dynamics system as% , ) Introducing momentum variable to form phase space , , , ), 、 Are respectively with The corresponding momentum vector is used to determine the momentum, Is a unit quaternion number, Scaling vectors for positive and designing adaptive momentum decay strategies for momentum vectors; s4, defining a rendering loss function Defining a rendering loss function as a comprehensive loss function formed by a rendering loss function, an edge loss function, an SSIM loss function and a multi-scale loss function; S5, constructing Hamiltonian volume Constructing hamiltonian volume according to hamiltonian dynamics system Deriving a regular equation according to the Hamiltonian quantity, namely acquiring the Hamiltonian equation, and continuously expressing the Hamiltonian equation; S6, optimizing process Constraint processing of unit quaternions using Leapfrog integrators And positive scaling vector And calculate And (3) with Then, carrying out self-adaptive setting on a quality matrix, and then carrying out random sub-sampling operation; s7 GPU parallel acceleration Updating the rendering and dynamics of each Gaussian ellipsoid into independent units by using a GPU; s8, mixing optimization strategy And (3) running a plurality of epochs by using a gradient descent method, optimizing a covariance matrix by using a Markov chain Monte Carlo algorithm, and finally evaluating the data by using the covariance matrix optimized by the Hamiltonian Monte Carlo algorithm in the steps S3-S7.
2. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 1, wherein in step S2, the covariance matrix is expressed as follows: ; Wherein, the A3 x 3 covariance matrix, which is a single gaussian ellipsoid, for describing the shape and direction of the gaussian ellipsoid; from unit quaternions A converted 3×3 rotation matrix; Is to scale vector with positive definite A related scale matrix.
3. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 1, wherein in step S3, the adaptive momentum decay strategy is specifically as follows: S31, calculating the change rate of the comprehensive loss function in the step S4 in the current iteration step, namely the ratio of the absolute value of the difference between the current loss and the loss in the last step to the corresponding time difference; S32, adjusting the momentum attenuation coefficient according to the change rate of the loss function, increasing the momentum attenuation coefficient when the change rate is smaller than a set threshold value and the convergence is fast, and decreasing the momentum attenuation coefficient when the change rate is larger than or equal to the set threshold value and the convergence is slow, wherein the update formula of the momentum attenuation coefficient is as follows: ; Wherein, the For the current momentum decay coefficient, Is to adjust the super-parameters of the device, Is the rate of change of the loss function, For the difference between the current loss and the previous loss, Representing the corresponding time difference value of the time difference, Is the updated momentum decay coefficient; s33, setting upper and lower limits of a momentum attenuation coefficient in order to prevent the momentum from being attenuated too fast or too slow; s34, updating and updating according to the new momentum attenuation coefficient Corresponding momentum vector 、。
4. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 1, wherein in step S4, the expression of the integrated loss function is as follows: ; Wherein, the To optimize the target, the potential energy item in Hamiltonian quantity is adopted; 、、、 Is the parameter of the ultrasonic wave to be used as the ultrasonic wave, The loss function is rendered on a basis, As a function of the edge loss, As a function of the SSIM loss, Is a multi-scale loss function; the base rendering loss function is defined as follows: ; Wherein, the The method is an image rendered by the current Gaussian ellipsoid parameters, namely a rendered image; Is a true image of the person, 、 The height and width of the rendered image and the real image respectively, Two-dimensional coordinates for rendering the image and real image pixels; The edge loss function is defined as follows: ; Wherein, the And Edge images of the rendered image and the real image respectively; The SSIM loss function is defined as follows: ; Wherein, the Representing the calculated structural similarity; The multiscale loss function is defined as follows: ; Wherein, the Representing the dimensions of the various dimensions of the device, Is the weight of each scale and, Is the pixel-by-pixel penalty per scale.
5. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 4, wherein in step S5, the expression of the hamiltonian amount is as follows: ; Wherein, the Is Hamiltonian, equal to the sum of potential energy and kinetic energy; 、 In the form of a quality matrix, 、 Representing its inverse matrix, the mass matrix is used to adjust the kinetic energy term.
6. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 5, wherein in step S5, the continuity of the hamiltonian equation is expressed as follows: ; ; ; ; Wherein, the Respectively are For a pair of 、 Is used for the gradient of (a), I.e. 。
7. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 6, wherein in step S6, when constraint processing is performed, for unit quaternions Re-regularization into unit quaternion after position update for positive scaling vector The algorithm is performed in the logarithmic domain, so that the update is performed in unconstrained space, and the positive domain is mapped again.
8. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 1 or 7, wherein in step S6, for unit quaternions when performing gradient calculations Original gradient Projected to a tangent space where a unit quaternion is located The gradient calculation is realized, and the specific processing mode is as follows: ; Wherein, the Is to project a vector onto a unit quaternion The tangential space operation in units of place ensures that the gradient is in a viable direction, I.e. , I.e. , For adjusting the factor, the calculation formula is as follows for adjusting the accuracy of the quaternion projection: ; Wherein, the Is a region Detail complexity, gamma and Is a super parameter, gamma is used for controlling the influence degree of detail complexity on projection accuracy, Is a constant term for setting the minimum projection accuracy; for positive scaling vectors Using logarithmic mapping, the gradient for the dynamics is calculated as follows: ; Wherein, as follows, the element-wise multiplication, i.e. matrix The elements in the two are multiplied correspondingly one by one, I.e. 。
9. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 1, wherein in the step S6, the quality matrix element is set to the inverse of the corresponding parameter history variance when the quality matrix is adaptively set.
10. The gaussian splatter covariance matrix optimization method based on the hamiltonian monte carlo algorithm according to claim 1, wherein in the step S8, a peak signal to noise ratio PSNR and a mean square error MSE are used as metrics for evaluating the covariance matrix optimization degree, wherein the mean square error Peak signal to noise ratio , For rendering image pixels The strength of the part is that of the part, For true image At pixel value MAX is the pixel maximum possible value.

Description

Gaussian splatter covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm Technical Field The invention relates to the technical field of 3D Gaussian splatter modeling, in particular to a Gaussian splatter covariance matrix optimization method based on a Hamiltonian Monte Carlo algorithm. Background Aiming at the defects of the existing optimization technology of covariance matrix in 3D Gaussian splats, the traditional 3D Gaussian splats adopt a gradient descent method to optimize Gaussian ellipsoid parameters of the covariance matrix, and are easy to sink into a local optimal solution. The covariance matrix needs to be kept semi-positive, but the direct optimization of gradient descent is easy to violate the constraint, so that the optimization difficulty of failure parameters in physical sense is increased, the convergence speed of standard gradient descent to high-dimensional parameters (such as 9-dimensional covariance) is low, and the anisotropic optimization is easy to fall into a sawtooth phenomenon, so that the reconstruction quality of a 3D scene is influenced. Covariance matrixRotation matrix R (quaternion) And scaling matrix S (vector) There is a strong nonlinear constraint (unit quaternion, positive quadty). Random gradient descent is sensitive to hyper-parameters, and covariance matrix degradation is prone to rendering distortion. Thereafter, related technicians use a markov chain monte carlo algorithm (such as a Metropolis-Hastings algorithm) to further optimize the covariance matrix, but such a method has low random walk efficiency in a parameter space, and the existing optimizers have difficulty in balancing the rendering precision (loss function non-convexity) and the calculation efficiency. The above problems need to be solved, and therefore, the invention provides an optimization and analysis method of a Gaussian splatter covariance matrix based on a Hamiltonian Monte Carlo algorithm. Disclosure of Invention The invention aims to solve the technical problems that a gradient descent method is easy to trap local optimum, violates semi-positive constraint, has low Markov chain Monte Carlo algorithm efficiency and the like, and provides a Gaussian splatter covariance matrix optimization method based on a Hamiltonian Monte Carlo algorithm. The invention solves the technical problems through the following technical proposal, and the invention comprises the following steps: S1, data acquisition and preprocessing Building image data are collected through the unmanned aerial vehicle, and are preprocessed, so that the building image data with high resolution are obtained; s2. covariance matrix construction Constructing a covariance matrix for describing the shape and direction of the Gaussian ellipsoid; s3, introducing momentum variable Constructing Hamiltonian dynamics system as%,) Introducing momentum variable to form phase space,,,),、Are respectively withThe corresponding momentum vector is used to determine the momentum,Is a unit quaternion number,Scaling vectors for positive and designing adaptive momentum decay strategies for momentum vectors; s4, defining a rendering loss function Defining a rendering loss function as a comprehensive loss function formed by a rendering loss function, an edge loss function, an SSIM loss function and a multi-scale loss function; S5, constructing Hamiltonian volume Constructing hamiltonian volume according to hamiltonian dynamics systemDeriving a regular equation according to the Hamiltonian quantity, namely acquiring the Hamiltonian equation, and continuously expressing the Hamiltonian equation; S6, optimizing process Constraint processing of unit quaternions using Leapfrog integratorsAnd positive scaling vectorAnd calculateAnd (3) withThen, carrying out self-adaptive setting on a quality matrix, and then carrying out random sub-sampling operation; s7 GPU parallel acceleration Updating the rendering and dynamics of each Gaussian ellipsoid into independent units by using a GPU; s8, mixing optimization strategy And (3) running a plurality of epochs by using a gradient descent method, optimizing a covariance matrix by using a Markov chain Monte Carlo algorithm, and finally evaluating the data by using the covariance matrix optimized by the Hamiltonian Monte Carlo algorithm in the steps S3-S7. Further, in the step S2, the covariance matrix is expressed as follows: ; Wherein, the A3 x 3 covariance matrix, which is a single gaussian ellipsoid, for describing the shape and direction of the gaussian ellipsoid; from unit quaternions A converted 3×3 rotation matrix; Is to scale vector with positive definite A related scale matrix. Further, in the step S3, the adaptive momentum decay strategy is specifically as follows: S31, calculating the change rate of the comprehensive loss function in the step S4 in the current iteration step, namely the ratio of the absolute value of the difference between the current loss and the loss in the last step to