CN-121999154-A - Illumination editing method, device, apparatus, storage medium and program product

CN121999154ACN 121999154 ACN121999154 ACN 121999154ACN-121999154-A

Abstract

The application discloses an illumination editing method, a device, equipment, a storage medium and a program product, which relate to the technical field of three-dimensional reconstruction, and comprise the steps of performing Gaussian splatter reconstruction on a scene image to obtain geometric representation of a target scene corresponding to the scene image; and merging the space shielding information and the ambient light source representation, and carrying out illumination rendering on the geometric representation to obtain an object image after illumination editing. By means of Gaussian splatter reconstruction of the scene, a foundation is laid for calculation of shielding information and environment light source representation, and the combination of fusion rendering of the shielding information and the environment light source enables illumination effects to follow the geometric structure of the scene and the light source direction to accurately and controllably change, ensures the sense of reality of illumination effects of edited images, achieves accurate control of illumination details, and improves the sense of reality and controllability of the edited illumination effects.

Inventors

WANG ZHE
QIAN YEQIANG
SONG YAWEI
GAN XIN
CAO DI
SANG CHEN
SUN YUE

Assignees

上汽通用五菱汽车股份有限公司

Dates

Publication Date: 20260508
Application Date: 20260112

Claims (10)

1. A light editing method, characterized in that the light editing method comprises: performing Gaussian splatter reconstruction on a scene image to obtain a geometric representation of a target scene corresponding to the scene image; Calculating spatial occlusion information and an ambient light source representation of the target scene based on the geometric representation; and fusing the space shielding information and the ambient light source representation, and performing illumination rendering on the geometric representation to obtain an object image after illumination editing.
2. The illumination editing method as set forth in claim 1, wherein the scene image includes a plurality of ambient background images defining an illumination environment, and wherein the step of calculating spatial occlusion information and an ambient light source representation of the target scene based on the geometric representation includes: setting a plurality of probe points in a scene space corresponding to the geometric representation; performing depth rendering to a plurality of surrounding view directions at each probe point to obtain a depth image at the probe point; Calculating a shielding coefficient at the probe point based on the depth image to obtain the space shielding information of the target scene; Splicing and converting a plurality of environmental background images into panoramic images; and encoding illumination information of the panoramic image by adopting spherical harmonic function to obtain an ambient light source representation represented by spherical harmonic coefficients.
3. The illumination editing method according to claim 2, wherein the step of calculating an occlusion coefficient at the probe point based on the depth image to obtain spatial occlusion information of the target scene includes: Based on a preset depth threshold value, comparing the depth image at the probe point to obtain a visibility graph at the probe point, wherein the visibility graph is used for representing the visibility of each view angle direction at the probe point; splicing the visibility graphs of the view directions at the same probe point to obtain a cube map at the probe point; Creating a virtual sphere at the probe point, and calculating a mapping relation between a three-dimensional coordinate system and a polar coordinate system based on the virtual sphere and the cube map, wherein the three-dimensional coordinate system is a coordinate system corresponding to the cube map, and the polar coordinate system is a coordinate system corresponding to the virtual sphere; mapping the pixel points in the cube map according to the mapping relation to obtain the direction vector and the solid angle weight of each pixel point on the virtual sphere; based on the direction vector and the solid angle weight, projecting the visibility graph by adopting a spherical harmonic basis, and calculating spherical harmonic basis values of the view angle directions at the probe points; And calculating the shielding coefficient at the probe point according to the spherical harmonic base value of each view angle direction to obtain the space shielding information of the target scene.
4. The illumination editing method as claimed in claim 2, wherein the method for stitching and converting a plurality of the environmental background images into a panoramic image comprises: Extracting and matching the characteristics of a plurality of environmental background images, and determining the relative pose relation between the environmental background images; Based on the relative pose relation, projecting a plurality of environmental background images into a panoramic coordinate system, and carrying out fusion processing on overlapping areas between adjacent images in the environmental background images to generate a panoramic image with a low dynamic range; And inputting the panoramic image with the low dynamic range into a pre-trained image conversion model, and converting the panoramic image with the low dynamic range into a panoramic image with the high dynamic range through the image conversion model.
5. The illumination editing method according to claim 2, wherein the step of setting a plurality of probe points in a scene space corresponding to the geometric representation comprises: Acquiring an axial bounding box of a scene point cloud corresponding to the geometric representation, and determining a calculation boundary of a scene space corresponding to the geometric representation by using the boundary of the axial bounding box; based on the calculated boundary, uniformly dividing the scene space corresponding to the geometric representation into regular three-dimensional grids by a preset voxel size, wherein one grid unit in the three-dimensional grids corresponds to one cube voxel; And setting probe points on the corner points of the cube voxels.
6. The illumination editing method according to claim 5, wherein the step of merging the spatial occlusion information and the ambient light source representation, and performing illumination rendering on the geometric representation to obtain an object image after illumination editing comprises the steps of: inquiring a target shielding coefficient of the target Gaussian point cloud on each corner point of a target voxel according to the three-dimensional space coordinates of the target Gaussian point cloud in the geometric representation, wherein the target voxel is a cube voxel corresponding to the target Gaussian point cloud, and the target Gaussian point cloud is any point in a scene point cloud corresponding to the geometric representation; calculating interpolation weights and backlight masks corresponding to all the corner points of the target voxels based on the relative position relation between the target Gaussian point cloud and all the corner points of the target voxels; Carrying out weighted summation on the backlight mask and the target shielding coefficient through the interpolation weight to obtain an interpolation coefficient corresponding to the target shielding coefficient; Performing function reconstruction on the ambient light source representation and the interpolation coefficient by adopting a spherical harmonic basis, and calculating the ambient illumination intensity emitted from the target Gaussian point cloud; Calculating the illumination color of the target Gaussian point cloud according to the reflectivity of the target Gaussian point cloud and the ambient illumination intensity; and carrying out illumination rendering on the target Gaussian point cloud according to the illumination color to obtain an object image after illumination editing.
7. An illumination editing apparatus, characterized in that the illumination editing apparatus comprises: the scene reconstruction module is used for performing Gaussian splatter reconstruction on a scene image to obtain the geometric representation of a target scene corresponding to the scene image; A parameter calculation module for calculating spatial occlusion information and an ambient light source representation of the target scene based on the geometric representation; And the rendering editing module is used for fusing the space shielding information and the ambient light source representation, and carrying out illumination rendering on the geometric representation to obtain an object image after illumination editing.
8. An illumination editing device, characterized in that the device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the illumination editing method according to any of claims 1 to 6.
9. A storage medium, characterized in that the storage medium is a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the illumination editing method according to any of claims 1 to 6.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when being executed by a processor, implements the steps of the illumination editing method as claimed in any of the claims 1 to 6.

Description

Illumination editing method, device, apparatus, storage medium and program product Technical Field The present application relates to the field of three-dimensional reconstruction technology, and in particular, to a method, apparatus, device, storage medium, and program product for illumination editing. Background Along with the rapid development of digital twin technology, high-precision three-dimensional scene reconstruction has become a core task for realizing high-fidelity scene restoration in the fields of intelligent driving and the like. At present, the reconstruction and rendering of the three-dimensional scene mainly comprises the steps of manually performing fine modeling and rendering through modeling software or realizing rapid reconstruction of the scene based on an automatic reconstruction algorithm. In order to meet the requirement of scene representation diversification, the above manner needs to support flexible editing of key environmental elements such as illumination conditions, wherein the editing of illumination intensity and direction is the key of illumination editing. However, the illumination editing mode based on the manual modeling needs to expend a great deal of effort to manually adjust the material properties of the scene model to simulate illumination change, and the process is complicated and highly dependent on manual experience, so that the physical rule is difficult to accurately control. More importantly, no matter in a manual mode or in most automatic reconstruction algorithms, the precise and efficient computing power of light shielding effect caused by complex geometric structures of a scene is generally lacking when illumination is edited. Particularly, under an outdoor scene, due to the fact that objects in the scene are numerous and complex in structure, light rays can be shielded by a large number of objects to form shadows in the propagation process, and an existing illumination editing mode is difficult to effectively model the shadows, so that the problems of insufficient sense of reality (such as hard shadows, missing shadows or geometrical shape incompatibility) and poor controllability (difficulty in accurately controlling the shapes and the softness of the shadows) exist in the edited illumination effect, and the high-standard scene visualization requirement cannot be met. The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present application and is not intended to represent an admission that the foregoing is prior art. Disclosure of Invention The application mainly aims to provide an illumination editing method, an illumination editing device, illumination editing equipment, a storage medium and a program product, and aims to solve the technical problems that an existing illumination editing mode is difficult to model effectively in an outdoor scene in a light propagation process, so that the edited illumination effect is insufficient in sense of reality and poor in controllability. In order to achieve the above object, the present application provides an illumination editing method, which includes: performing Gaussian splatter reconstruction on a scene image to obtain a geometric representation of a target scene corresponding to the scene image; Calculating spatial occlusion information and an ambient light source representation of the target scene based on the geometric representation; and fusing the space shielding information and the ambient light source representation, and performing illumination rendering on the geometric representation to obtain an object image after illumination editing. In an embodiment, the scene image comprises a plurality of ambient background images defining a lighting environment, the step of computing spatial occlusion information and an ambient light source representation of the target scene based on the geometric representation comprising: setting a plurality of probe points in a scene space corresponding to the geometric representation; performing depth rendering to a plurality of surrounding view directions at each probe point to obtain a depth image at the probe point; Calculating a shielding coefficient at the probe point based on the depth image to obtain the space shielding information of the target scene; Splicing and converting a plurality of environmental background images into panoramic images; and encoding illumination information of the panoramic image by adopting spherical harmonic function to obtain an ambient light source representation represented by spherical harmonic coefficients. In an embodiment, the step of calculating the occlusion coefficient at the probe point based on the depth image to obtain the spatial occlusion information of the target scene includes: Based on a preset depth threshold value, comparing the depth image at the probe point to obtain a visibility graph at the probe point, wherein the visibility graph is used for represen