CN-121982220-A - Cloud Gaussian splatter scene automatic generation and multi-terminal distribution method

CN121982220ACN 121982220 ACN121982220 ACN 121982220ACN-121982220-A

Abstract

The invention discloses an automatic generation and multi-terminal distribution method of a cloud Gaussian splash scene, which belongs to the technical field of three-dimensional model generation and cloud distribution, and comprises the steps of receiving a single object video and user parameters; the method comprises the steps of performing self-adaptive frame extraction on video, preprocessing to obtain an image sequence with a transparent channel, performing alignment reconstruction through a double alignment engine, defining quality indexes, performing Gaussian splatter training after standardized verification of reconstruction output, deriving a result in an iteration stage to form an LOD model set, merging alignment and training quality indexes to obtain comprehensive scores, performing threshold judgment or self-adaptive optimal scheduling according to the scores, and finally packaging the model into an asset package to be distributed according to terminal capacity.

Inventors

LI BING
LIU YILI
CHEN XIANJUN
CHEN BO
LI YUAN
SHENG JIANWEN

Assignees

元梦空间文化传播(成都)有限公司

Dates

Publication Date: 20260505
Application Date: 20260212

Claims (10)

1. The cloud Gaussian splatter scene automatic generation and multi-terminal distribution method is characterized by comprising the following steps of: S1, receiving a single object video and user parameters uploaded by a user; s2, decoding the single object video, adaptively extracting frames to generate an image sequence, and preprocessing the frame extraction result to obtain the image sequence with the transparent channel; S3, performing alignment reconstruction based on the preprocessed image sequence, outputting internal and external parameters of the camera, image matching relation and sparse structure information, and defining quality indexes of an alignment reconstruction stage; S4, converting the aligned reconstruction output into a trainable data set structure and executing standardized verification; s5, performing Gaussian splash scene training based on the standardized data set, and deriving training results in stages to form a hierarchical detail model set; S6, defining a training quality index of the training step S5, weighting the quality index of the alignment reconstruction stage and the training quality index after normalization treatment to obtain a comprehensive quality score Q, comparing and judging the Q with a preset threshold value Q min , skipping to the next step when the preset condition is met, and executing self-adaptive optimized scheduling and repeating the comparison and judgment when the preset condition is not met until the preset condition is met or a preset upper limit condition is reached; and S7, packaging the trained hierarchical detail model set into a distributable asset package, generating an asset list, and selecting corresponding configuration issuing model assets to the terminal according to the terminal capability.
2. The method for automatically generating and distributing cloud gaussian splatter scenes according to claim 1, wherein in S1, the user parameters include a desired quality level, a maximum gaussian point constraint, and an output configuration, wherein the preset threshold Q min corresponds to the desired quality level, the maximum gaussian point constraint is used as a quantization control basis in the training stage of S5, and the output configuration provides an execution basis for S7.
3. The cloud Gaussian splatter scene automatic generation and multi-terminal distribution method according to claim 1, wherein in S2, adaptive frame extraction is that frame extraction intervals are determined according to video frame rate, target motion amplitude and definition, preprocessing sequentially comprises definition, exposure, noise screening, main body separation, robust enhancement facing reflective and transparent materials, and the robust enhancement is that a low confidence region is generated for a specular reflection or transparent region.
4. The cloud gaussian splatter scene automation generating and multi-terminal distribution method of claim 1, wherein in S3, a first alignment engine and a second alignment engine are configured to execute alignment reconstruction, the alignment reconstruction is performed by selecting one or a combination of the first alignment engine and the second alignment engine, and the quality index of the alignment reconstruction stage comprises: ratio of number of aligned successful images to total number of images: ; reprojection error statistics: ; Coverage density index of sparse structure: ; Wherein omega is an effective alignment set, K i ,R i ,T i is the rotation and translation of the internal parameter and the external parameter of the ith camera respectively, X j is the j three-dimensional point coordinate, X ij is the observed value of the pixel coordinate of the characteristic point on the ith image, and pi is # ) N pt is the number of points in the sparse structure, which is the projection function.
5. The automated cloud gaussian splatter scene generation and multi-terminal distribution method according to claim 1, wherein in S4, the standardized check comprises mapping a coordinate system and a scale to a training unified specification, mapping a camera model and a parameter to a training unified format, abnormal observation rejection, consistency check and repair.
6. The cloud gaussian splatter scene automatic generation and multi-terminal distribution method according to claim 4, wherein in S6, the training quality index comprises a training loss reduction amount: ; training convergence judgment results: ; Wherein L t is the training loss of the t step, k is the preset step number interval, To indicate the function, the function is indicated The definition is as follows: 。
7. The automated cloud gaussian splatter scene generation and multi-terminal distribution method according to claim 6, wherein in S6, the quality index of the alignment reconstruction stage and the training quality index are functionally related Normalizing to obtain mass fractions of each dimension, wherein: Camera alignment score: ; Reprojection error score: ; Sparse structure density fraction: ; Comprehensive quality scoring: ; Wherein the method comprises the steps of In order to preset the threshold value parameter(s), Is non-negative weight and satisfies 。
8. The cloud gaussian splatter scene automatic generation and multi-terminal distribution method according to claim 7, wherein in S6, the preset conditions are: the process proceeds to the next step in time, Executing self-adaptive optimal scheduling; Control action set of adaptive optimized scheduling The method comprises retry and parameter adjustment, alignment engine switching, rollback and increment repair, breakpoint continuous running and cache multiplexing, and the control action selection rules are as follows: ; Wherein, the To generate a quality score corresponding after performing action u, The control action set is the closed loop deviation rectifying control action set.
9. The cloud gaussian splatter scene automatic generation and multi-terminal distribution method according to claim 8, wherein in S6, the adaptive optimal scheduling is terminated when a preset upper limit of resources or time is reached and a termination state is outputted.
10. The cloud gaussian splatter scene automatic generation and multi-terminal distribution method according to claim 1, wherein in S7, an asset list at least comprises asset version information and verification information, LOD hierarchical entries, terminal adaptation configuration and loading information, the LOD hierarchical entries map models derived from different iteration steps into different hierarchical detail levels, the terminal adaptation configuration is configured as a mobile terminal, web, PC, VR platforms respectively specify default LOD levels, loading strategies and resource constraint parameters, the loading information supports on-demand and streaming loading, the asset distribution selects the LOD levels and distribution strategies under corresponding configurations according to terminal capabilities and the terminal analyzes and loads and displays the asset list.

Description

Cloud Gaussian splatter scene automatic generation and multi-terminal distribution method Technical Field The invention belongs to the technical field of three-dimensional model generation and cloud distribution, and particularly relates to an automatic generation and multi-terminal distribution method for a cloud Gaussian splash scene. Background In the prior art, a process of generating a Gaussian splatter model through video or image training generally needs to undergo steps of frame extraction, background elimination, alignment/reconstruction, training, derivation and the like, the process needs to be processed by means of a plurality of software tools or scripts, and the process depends on engineering personnel to manually execute operations, check intermediate results, modify configuration and repeatedly run again and again, so that the whole process is strong in dependence on experience, complex in steps and large in result fluctuation. When providing cloud service to a general user, the manual processing method can lead to uncontrollable process, extremely low efficiency and difficult scale, and has the following defects: The method cannot be automated and depends on manual and strong experience, wherein links such as a frame extraction strategy, image matting parameters, alignment configuration, training parameters and the like are manually and repeatedly adjusted and manually executed, and when a certain step fails or is abnormal, the process is re-executed by manually positioning reasons, and the process is uncontrollable, low in efficiency, high in cost and difficult to deploy in a large scale; The existing process can only ensure that the process can be executed, and a quantitative evaluation and threshold triggering mechanism of the output quality of intermediate steps is not needed, so that the failure rate is high, the reworking is more, and the cloud computing cost is wasted; The output result of multi-terminal adaptation is lacking, namely the training result is deeply bound with the output format of training software, multi-model format support and hierarchical detail (LOD) design are not available, and the configuration of different terminals such as a mobile terminal, web, PC, VR and the like is not available, so that the cross-terminal use cost is high and the display consistency is difficult to ensure; The existing flow has insufficient support for multiplexing intermediate results, failed retries and breakpoint recovery, is easy to cause repeated calculation, wastes calculation resources and increases processing time. Disclosure of Invention In order to solve the problems in the background art, the invention provides an automatic generation and multi-terminal distribution method of a cloud Gaussian splatter scene, which aims to solve the problems of low automation degree, uncontrollable quality, poor cross-terminal adaptability and waste of computing resources of the existing Gaussian splatter model generation flow. In order to achieve the above purpose, the present invention provides the following technical solutions: a cloud Gaussian splatter scene automatic generation and multi-terminal distribution method comprises the following steps: S1, receiving a single object video and user parameters uploaded by a user; s2, decoding the single object video, adaptively extracting frames to generate an image sequence, and preprocessing the frame extraction result to obtain the image sequence with the transparent channel; S3, performing alignment reconstruction based on the preprocessed image sequence, outputting internal and external parameters of the camera, image matching relation and sparse structure information, and defining quality indexes of an alignment reconstruction stage; S4, converting the aligned reconstruction output into a trainable data set structure and executing standardized verification; s5, performing Gaussian splash scene training based on the standardized data set, and deriving training results in stages to form a hierarchical detail model set; S6, defining a training quality index of the training step S5, weighting the quality index of the alignment reconstruction stage and the training quality index after normalization treatment to obtain a comprehensive quality score Q, comparing and judging the Q with a preset threshold value Q min, skipping to the next step when the preset condition is met, and executing self-adaptive optimized scheduling and repeating the comparison and judgment when the preset condition is not met until the preset condition is met or a preset upper limit condition is reached; and S7, packaging the trained hierarchical detail model set into a distributable asset package, generating an asset list, and selecting corresponding configuration issuing model assets to the terminal according to the terminal capability. Preferably, in S3, an alignment reconstruction is performed by configuring the first alignment engine and the second alignment engine, and