CN-121999110-A - Novel view angle synthesis method based on double-view angle guiding three-dimensional Gaussian sputtering

Abstract

The invention relates to the technical field of computer vision graphics and nerve rendering, in particular to a novel view angle synthesis method of three-dimensional Gaussian sputtering based on double-view angle guidance, which has the technical scheme that a main view angle pair and an auxiliary view angle pair are selected for joint optimization in each round of training, the attention of common view anchor point characteristics is enhanced, and meanwhile, multi-view projection consistency loss and multi-scale strength loss are introduced, so that the overall structural stability and the local texture detail quality are improved; the method comprises the steps of training the pair of visual angles through high overlapping and moderate parallax, fully utilizing multi-visual angle complementary information, reducing single-visual angle bias, enhancing the attention of common-view anchor points, explicitly modeling the relationship among the anchor points, improving the expression capability of local textures and structures, inhibiting anchor point drift through projection consistency loss, improving detail and contrast structures through multi-scale strength loss, improving rendering quality and enhancing geometric consistency, and solving the problems of poor geometric consistency, unstable anchor points and detail artifact caused by the existing single-visual angle training.

Inventors

• TANG XIANGYAN
• HUANG HAIPING

Assignees

• 海南大学

Dates

Publication Date

20260508

Application Date

20260128

Claims (10)

1. 1. The new view angle synthesizing method based on the double-view angle guiding three-dimensional Gaussian sputtering is characterized by comprising the following steps: S1, scene initialization and anchor point representation construction, namely acquiring a multi-view input image and camera parameters, initializing a three-dimensional Gaussian according to a sparse point cloud or constructing an anchor point set based on a voxel grid, wherein each anchor point carries a feature vector and a plurality of offsets to generate a nerve Gaussian; s2, selecting an auxiliary view angle from the rest view angles according to the current main view angle in each round of training, so as to perform main and auxiliary view angle matching; S3, marking a double-view joint training and a common-view anchor point, namely, based on the selected main and auxiliary view pairs, jointly executing one training iteration, marking an anchor point or a nerve Gaussian set commonly visible under two view angles, and using the anchor point or the nerve Gaussian set for subsequent cross-view consistency constraint and feature enhancement; S4, common view anchor point attention enhancement, wherein an anchor point characteristic enhancement module aggregates common view anchor point characteristic vectors, uses an attention mechanism to model deep dependency relations among anchor points and fuses geometric and appearance information of main and auxiliary view angles; S5, constructing a loss function and updating parameters, namely constructing a total loss at least comprising two types of loss of multi-view projection consistency loss and multi-scale intensity loss and updating the parameters; And S6, new view angle rendering output, namely performing micro-rasterization rendering of the new view angle by using the optimized anchor points or the nerve Gaussian parameters after training is finished, and outputting a new view angle synthesized image.
2. 2. The method of claim 1, wherein in S1, the properties of the nerve Gaussian include at least color, opacity, rotation and scaling.
3. 3. The novel view angle synthesizing method based on the double-view angle guiding three-dimensional Gaussian sputtering is characterized in that in S2, the selection principle of the main view angle and the auxiliary view angle is simultaneously satisfied, the main view angle and the auxiliary view angle have visible anchor points or point cloud overlapping and have parallax, the view angle pair with the highest score can be selected through a scoring function of the visibility relevance and the view angle difference penalty, and the view angle pair outside the set parallax range is filtered.
4. 4. The new view angle synthesizing method based on the dual view angle guided three-dimensional Gaussian sputtering of claim 1, wherein in S4, the anchor point feature enhancement module only acts on a training optimization stage.
5. 5. The new view angle synthesizing method based on the dual view angle guided three-dimensional Gaussian sputtering of claim 1, wherein in S5, the multi-view projection consistency loss constrains the projection positions of the same anchor point under the main and auxiliary view angles.
6. 6. The new view angle synthesizing method based on the dual view angle guided three-dimensional Gaussian sputtering of claim 1, wherein in S5, the multi-scale intensity loss converts the rendered image and the real image into a brightness image, the brightness is aligned under the multi-scale, and meanwhile, the brightness edge gradient is extracted and the edge consistency is calculated.
7. 7. The new view angle synthesizing method based on the double-view-angle guided three-dimensional Gaussian sputtering of claim 1, wherein in S1, after a multi-view-angle input image set and camera inner-outer parameters thereof are acquired, a projection operator of each view angle is calculated.
8. 8. The new view angle synthesizing method based on the dual view angle guided three-dimensional Gaussian sputtering of claim 1, wherein in S1, each anchor point comprises a three-dimensional position and a learnable feature vector, a plurality of learnable offset vectors are set for each anchor point, an attribute prediction network is constructed, and a micro renderer is initialized to execute Gaussian sputtering rasterization to obtain a synthesized image.
9. 9. The new view angle synthesizing method based on the double view angle guided three-dimensional Gaussian sputtering of claim 1 is characterized in that in S4, common view anchor point features are taken and input into an attention enhancing module to model dependency relations among anchor points, and enhanced features are obtained.
10. 10. The new view angle synthesizing method based on the dual view angle guided three-dimensional Gaussian sputtering of claim 1, wherein between S1 and S2, the visibility mask and view angle correlation matrix construction and the view angle difference calculation are carried out.

Description

Novel view angle synthesis method based on double-view angle guiding three-dimensional Gaussian sputtering Technical Field The invention relates to the technical field of computer vision graphics and nerve rendering, in particular to a novel view angle synthesis method based on double-view angle guiding three-dimensional Gaussian sputtering. Background The new view synthesis task typically generates an unobserved view image based on the multi-view input image. In recent years, three-dimensional Gaussian sputtering (3D Gaussian Splatting,3DGS) has been shown to be prominent in terms of rendering quality and speed, but its training paradigm has been followed by single view supervision, where each iteration randomly selects one view and only the view is used to reconstruct the error update parameters. The strategy easily causes excessive deviation of model parameters to a training view angle, so that oscillation, drift and redundancy of a three-dimensional Gaussian or anchor point occur under multiple view angles, and finally, detail artifacts and unreasonable floating ellipsoids are generated in a rendering result, and the consistency of a bottom structure is insufficient. In addition, the anchor point 3DGS method (e.g., introducing anchor points and nerve gauss through voxel grids, predicting nerve gauss attributes using MLP) can alleviate gaussian overstretching and improve structural distribution, but if single view training is still adopted, it is still difficult to fully utilize multi-view complementary information, especially under-supervision under occlusion, blurring or low quality view. In view of the above, we propose a new view angle synthesis method based on dual view angle guided three-dimensional gaussian sputtering to solve the existing problems. Disclosure of Invention The invention aims to provide a novel view angle synthesizing method based on double-view angle guiding three-dimensional Gaussian sputtering, which aims to solve the problems in the background art. In order to achieve the purpose, the invention provides a novel view angle synthesizing method based on double-view angle guiding three-dimensional Gaussian sputtering, which comprises the following steps: S1, scene initialization and anchor point representation construction, namely acquiring a multi-view input image and camera parameters, initializing a three-dimensional Gaussian according to a sparse point cloud or constructing an anchor point set based on a voxel grid, wherein each anchor point carries a feature vector and a plurality of offsets to generate a nerve Gaussian; s2, selecting an auxiliary view angle from the rest view angles according to the current main view angle in each round of training, so as to perform main and auxiliary view angle matching; S3, marking a double-view joint training and a common-view anchor point, namely, based on the selected main and auxiliary view pairs, jointly executing one training iteration, marking an anchor point or a nerve Gaussian set commonly visible under two view angles, and using the anchor point or the nerve Gaussian set for subsequent cross-view consistency constraint and feature enhancement; S4, common view anchor point attention enhancement, wherein an anchor point characteristic enhancement module aggregates common view anchor point characteristic vectors, uses an attention mechanism to model deep dependency relations among anchor points and fuses geometric and appearance information of main and auxiliary view angles; S5, constructing a loss function and updating parameters, namely constructing a total loss at least comprising two types of loss of multi-view projection consistency loss and multi-scale intensity loss and updating the parameters; And S6, new view angle rendering output, namely performing micro-rasterization rendering of the new view angle by using the optimized anchor points or the nerve Gaussian parameters after training is finished, and outputting a new view angle synthesized image. Further, in S1, the attributes of the nerve gauss include at least color, opacity, rotation, scaling. Further, in S2, the selection principle of the main and auxiliary view angles simultaneously meets the conditions that the main and auxiliary view angles have visible anchor points or point cloud overlapping and have parallax, and the view angle pair with the highest score can be selected through a scoring function of the visibility relevance and view angle difference penalty, and the view angle pair outside the parallax range is filtered. Further, in S4, the anchor feature enhancement module only acts on the training optimization stage. Further, in S5, the multi-view projection consistency penalty constrains the projection positions of the same anchor point at the primary and secondary views. Further, in S5, the multi-scale intensity loss converts the rendered map and the real map into luminance maps, aligns the luminance at multiple scales, extracts luminance edge gradients and c