CN-121999144-A - Three-dimensional reconstruction method based on Gaussian point information redistribution and geometric structure constraint

CN121999144ACN 121999144 ACN121999144 ACN 121999144ACN-121999144-A

Abstract

The application relates to a three-dimensional reconstruction method based on Gaussian point information redistribution and geometric structure constraint. The method comprises the steps of obtaining target scene image data, extracting a visual Gaussian point attribute parameter set based on a 3D Gaussian sputtering frame, randomly removing part of Gaussian points, determining a neighborhood point set, constructing an approximate model to obtain a gradient item set, constructing an association matrix according to the gradient item set, calculating an information distribution coefficient, compensating opacity and color information of the removed points to the neighborhood points to update the parameter set, constructing a total loss function containing basic reconstruction, local direction consistency and main direction alignment loss, iterating optimization parameters, dynamically adjusting the Gaussian point shielding rate, modifying the opacity association parameter, maintaining physical constraint, repeating information redistribution until training is completed, and realizing high-quality three-dimensional reconstruction. The method can improve the definition of the geometric structure.

Inventors

LIU XIAOHUI
Yuan Yuelin
Yin Tingli
XU GUIRONG
LIAO CHENZHONG
WEN CHAO

Assignees

中国人民解放军国防科技大学

Dates

Publication Date: 20260508
Application Date: 20260409

Claims (10)

1. A three-dimensional reconstruction method based on gaussian point information redistribution and geometric structure constraints, the method comprising: Acquiring image data of a target scene under a sparse view angle, acquiring a visible Gaussian point set covering pixel points from the image data based on a three-dimensional Gaussian sputtering frame, extracting attribute parameters of each Gaussian point from the visible Gaussian point set, and forming a Gaussian point attribute parameter set; Randomly eliminating a preset number of Gaussian points from the visual Gaussian point set to serve as Gaussian points to be eliminated, and determining a neighborhood Gaussian point set corresponding to each Gaussian point to be eliminated in the visual Gaussian point set; Based on the Gaussian point attribute parameter set, constructing an approximate model of the pixel point color with respect to the opacity of the Gaussian points, and calculating to obtain gradient items of the pixel point color with respect to the opacity of each Gaussian point to form a gradient item set; Based on the gradient item set, constructing an incidence matrix of each Gaussian point to be removed and the Gaussian point in the corresponding neighborhood Gaussian point set, and calculating an information distribution coefficient of each neighborhood Gaussian point according to the incidence matrix and the information distribution factor to form an information distribution coefficient set; According to the information distribution coefficient set, compensating the opacity and color information of each Gaussian point to be removed to the corresponding Gaussian point in the neighborhood Gaussian point set, and updating the opacity and color attribute parameters of the neighborhood Gaussian point to obtain an updated Gaussian point attribute parameter set; constructing a total loss function of geometric structure constraint, wherein the total loss function comprises basic reconstruction loss, local direction consistency loss and main direction alignment loss, and performing iterative optimization on an updated Gaussian point attribute parameter set based on the total loss function; In the iterative optimization process, the Gaussian point shielding rate is gradually and dynamically adjusted, the associated parameters of the opacity of the Gaussian points are modified based on the adjusted Gaussian point shielding rate, the physical constraint interval of the opacity is maintained, meanwhile, the Gaussian point information of the round is redistributed based on the Gaussian point attribute parameter set after iterative optimization until the preset training iteration times are completed, the finally optimized Gaussian point attribute parameter set is obtained, and the three-dimensional reconstruction of the target scene is completed based on the finally optimized Gaussian point attribute parameter set.
2. The method of claim 1, wherein the Gaussian point attribute parameter set comprises a color of a pixel point, the color of the pixel point being Wherein, the Indicating the opacity of the ith gaussian point at pixel point p, Representing Gaussian point at pixel The projection weight at which the image is projected, Represent the first The pixel coordinates of the center of projection of the gaussian point at the current viewing angle, Representing pixels The coordinates of the pixels are used to determine, As a scale parameter approximated by the post-projection covariance, For the color characteristic of the ith gaussian point at pixel point p, Expressed as the transmittance of the ith gaussian point at the pixel point p, K denotes the number of gaussian points covering the pixel point p.
3. The method of claim 1, wherein constructing an approximation model of pixel point color with respect to gaussian point opacity based on the gaussian point property parameter set comprises: based on the Gaussian point attribute parameter set, constructing an approximate model of pixel point colors relative to the opacity of the Gaussian points as follows Wherein, the The color of the pixel point is indicated, Indicating the opacity of the ith gaussian point at pixel point p, Representing Gaussian point at pixel The projection weight at which the image is projected, Represent the first The pixel coordinates of the center of projection of the gaussian point at the current viewing angle, Representing pixels The coordinates of the pixels are used to determine, As a scale parameter approximated by the post-projection covariance, For the color characteristic of the ith gaussian point at pixel point p, Representing the transmittance of the ith gaussian point at pixel point p, A gradient term indicating that the color at pixel point p is opaque to the ith gaussian point.
4. The method of claim 1, wherein computing gradient terms of the opacity of the pixel point color to each gaussian point to form a set of gradient terms, comprising: combining Gao Sidian projection weights at the pixel points, and calculating to obtain gradient items of the opacity of the pixel point colors to each Gaussian point through the approximate model; And integrating the gradient items of all the Gaussian points according to the corresponding relation between the Gaussian points and the pixel points to form the gradient item set.
5. The method of claim 1, wherein constructing an association matrix of each gaussian point to be culled with a gaussian point in a corresponding neighborhood gaussian point set based on the gradient term set comprises: based on the gradient item set, constructing an incidence matrix of each Gaussian point to be removed and the Gaussian point in the corresponding neighborhood Gaussian point set as follows Wherein, the Representing the transmittance of the ith gaussian point at pixel point p, Representing the ith Gaussian point at the pixel The projection weight at which the image is projected, The transmittance of the jth gaussian point at the pixel point p is represented, Representing the jth Gaussian point at the pixel The projection weight at which the image is projected, Representing the color characteristics of the ith gaussian point, Representing the color characteristics of the jth gaussian point.
6. The method of claim 1, wherein calculating the information distribution coefficient for each neighborhood gaussian point based on the correlation matrix and the information distribution factor comprises: Calculating the information distribution coefficient of each neighborhood Gaussian point according to the incidence matrix and the information distribution factor to obtain the information distribution coefficient of each neighborhood Gaussian point as Wherein, the The information allocation factor is represented by a number of information, Representing the correlation matrix of the image, Representing the inverse of the correlation matrix.
7. The method of claim 1, wherein compensating the opacity and color information of each gaussian point to be removed to the gaussian point in the corresponding neighborhood gaussian point set according to the information distribution coefficient set, updating the opacity and color attribute parameters of the neighborhood gaussian point, and obtaining an updated gaussian point attribute parameter set, comprises: According to the information distribution coefficient set, the opacity and the color information of each Gaussian point to be removed are compensated to the corresponding Gaussian point in the neighborhood Gaussian point set, the opacity and the color attribute parameters of the neighborhood Gaussian point are updated, and the updated Gaussian point attribute parameters are respectively Wherein, the 、 The opacity and color characteristics of the previous neighborhood gaussian point i are updated respectively, 、 The opacity and color characteristics of the gaussian point j to be culled, Assigning coefficients to the information of the neighborhood gaussian point i, 、 Is the updated opacity and color characteristics of the neighborhood gaussian point i.
8. The method of claim 1, wherein constructing the overall loss function of the geometry constraint is , wherein, The L1 penalty for rendering the image and the original image plus the structural similarity penalty, 、 The weight super-parameters of the local direction consistency loss and the main direction alignment loss, In the event of a loss of local directional consistency, Alignment loss for the main direction.
9. The method of claim 8, wherein the local directional consistency penalty is ; The main direction alignment loss is Wherein, the Indicating the number of all gaussian points falling in the edge region, And The unit direction vector from the current point to the nearest first and second points, Is the main direction unit vector of the current Gaussian point, is obtained by the covariance matrix, Represents a unit direction vector formed by the current point and the adjacent point, The number of the neighboring points is set.
10. The method of claim 1, wherein modifying the associated parameters of gaussian point opacity based on the adjusted gaussian point mask rate comprises: Gaussian point opacity The corresponding relation with the associated parameter tau is: based on Gaussian point shielding rate The process of modifying the association parameter τ is: Where τ is the associated parameter before modification, Is an associated parameter of modified gaussian point opacity.

Description

Three-dimensional reconstruction method based on Gaussian point information redistribution and geometric structure constraint Technical Field The application relates to the technical field of computer vision and three-dimensional reconstruction, in particular to a three-dimensional reconstruction method based on Gaussian point information redistribution and geometric structure constraint. Background The three-dimensional reconstruction technology aims at recovering the three-dimensional geometric structure and texture information of the real world from the observed data, and has important application value in the fields of robot perception, automatic driving, virtual reality and the like. The three-dimensional Gaussian sputtering (3 DGS) technology has the advantages of both reconstruction quality and efficiency, and is one of the mainstream technologies. At sparse viewing angles, the 3DGS technique has two core problems. Firstly, the geometric structure is fuzzy, and only through two-dimensional image supervision training, the spatial distribution of the Gaussian point cloud is unclear, so that the subsequent tasks of three-dimensional target recognition, point cloud semantic segmentation and the like are influenced. Secondly, the training data is insufficient due to the fact that the input visual angle is limited after fitting, floating artifacts are prone to occur, and the new visual angle rendering effect is poor. The prior method improves the geometric structure by introducing depth information or edge supervision, but the boundary of the depth map is fuzzy, so that the detail expression capability is weakened, and the overfitting is relieved by randomly shielding Gaussian points, but the problems of Gaussian point redundancy and information loss are not solved. Disclosure of Invention In view of the foregoing, it is desirable to provide a three-dimensional reconstruction method based on gaussian point information redistribution and geometric structure constraint that can improve the definition of geometric structures. A three-dimensional reconstruction method based on gaussian point information redistribution and geometric structure constraints, the method comprising: Acquiring image data of a target scene under a sparse view angle, acquiring a visible Gaussian point set covering pixel points from the image data based on a three-dimensional Gaussian sputtering frame, and extracting attribute parameters of each Gaussian point from the visible Gaussian point set to form a Gaussian point attribute parameter set; randomly eliminating a preset number of Gaussian points from the visual Gaussian point set to serve as Gaussian points to be eliminated, and determining a neighborhood Gaussian point set corresponding to each Gaussian point to be eliminated in the visual Gaussian point set; Based on the Gaussian point attribute parameter set, constructing an approximate model of the pixel point color with respect to the opacity of the Gaussian points, and calculating to obtain gradient items of the opacity of the pixel point color to each Gaussian point to form a gradient item set; Based on the gradient item set, constructing an incidence matrix of each Gaussian point to be removed and the Gaussian point in the corresponding neighborhood Gaussian point set, and calculating an information distribution coefficient of each neighborhood Gaussian point according to the incidence matrix and the information distribution factor to form an information distribution coefficient set; according to the information distribution coefficient set, compensating the opacity and color information of each Gaussian point to be removed to the Gaussian point in the corresponding neighborhood Gaussian point set, and updating the opacity and color attribute parameters of the neighborhood Gaussian point to obtain an updated Gaussian point attribute parameter set; constructing a total loss function of geometric structure constraint, wherein the total loss function comprises basic reconstruction loss, local direction consistency loss and main direction alignment loss, and performing iterative optimization on the updated Gaussian point attribute parameter set based on the total loss function; In the iterative optimization process, the Gaussian point shielding rate is adjusted gradually and dynamically, the associated parameters of the opacity of the Gaussian points are modified based on the adjusted Gaussian point shielding rate, the physical constraint interval of the opacity is maintained, meanwhile, the Gaussian point information of the round is redistributed based on the Gaussian point attribute parameter set after iterative optimization until the preset training iteration times are completed, the finally optimized Gaussian point attribute parameter set is obtained, and the three-dimensional reconstruction of the target scene is completed based on the finally optimized Gaussian point attribute parameter set. According to the three-dimensio