Search

CN-122023479-A - Camera shake image data set generation method, device, electronic equipment and medium

CN122023479ACN 122023479 ACN122023479 ACN 122023479ACN-122023479-A

Abstract

The invention relates to the technical field of computer vision and graphics, in particular to a method, a device, electronic equipment and a medium for generating a camera shake image data set, wherein the method comprises the following steps of obtaining at least one 3D scene model, shake track data and lens parameters of a camera and generating a configuration file corresponding to each rendering flow; based on the configuration files corresponding to each rendering process, the rendering process is constructed, each 3D scene model is processed according to a preset dynamic camera shake rendering method to obtain a dynamic blurred image, each 3D scene model is processed according to a preset still camera rendering method to obtain a still image, and a camera shake image dataset is obtained based on the dynamic blurred image and the still image, so that the problems that the difference between a blurred image and an actual image generated in the related technology is overlarge and the physical reality is insufficient are solved, and the reality and the physical consistency of the camera shake image dataset are improved.

Inventors

  • JI XIANGYANG
  • LI YUANHANG
  • WEI HENGLU

Assignees

  • 清华大学

Dates

Publication Date
20260512
Application Date
20260113

Claims (10)

  1. 1. A camera shake image dataset generation method, characterized by comprising the steps of: Acquiring at least one 3D scene model, shake locus data of a camera and lens parameters, and generating a configuration file corresponding to each rendering flow; And constructing a rendering process based on the configuration file corresponding to each rendering process, processing each 3D scene model according to a preset dynamic camera shake rendering method to obtain a dynamic blurred image, processing each 3D scene model according to a preset still camera rendering method to obtain a still image, and obtaining a camera shake image data set based on the dynamic blurred image and the still image.
  2. 2. The method of claim 1, wherein the obtaining at least one scene model, shake trajectory data of a camera, and lens parameters, generating a configuration file corresponding to each rendering flow, comprises: acquiring 3D scene model information, camera related data and lens parameters corresponding to each scene; Extracting illumination information, background information, 3D scene model parameters and rendering control parameters when each 3D scene model is generated based on scene model information, camera related data and lens parameters corresponding to each scene; and acquiring the space coordinates of the object objects in the image configuration file corresponding to each 3D scene model, and determining the surface optical model corresponding to the 3D scene model according to the space coordinates of the object objects in the image configuration file corresponding to each 3D scene model.
  3. 3. The method according to claim 1, wherein the constructing a rendering process based on the configuration file corresponding to each rendering process, processing each 3D scene model according to a preset dynamic camera shake rendering method to obtain a dynamic blurred image, processing each 3D scene model according to a preset still camera rendering method to obtain a still image, and obtaining a camera shake image dataset based on the dynamic blurred image and the still image, includes: constructing a physical lens model based on lens parameters in an initialized image configuration file of each 3D scene model, and determining an entrance pupil position, an exit pupil position and distortion mapping of the physical lens model according to a preset ray tracing method; Acquiring an actual camera shake state corresponding to each rendering time point in the camera shake track data, obtaining a pose sequence of the camera according to the actual camera shake state corresponding to each rendering time point, and carrying out interpolation calculation on the camera pose in the pose sequence based on a preset interpolation processing algorithm to obtain a continuous dynamic pose of the camera in the rendering process; Determining a spatial view angle of each moment camera based on the continuous dynamic pose of the camera, generating a plurality of rays of each pixel in each 3D scene model based on the spatial view angle of each moment camera, and determining scattering points of the surface of each object in each 3D scene model according to the plurality of rays of each pixel in each 3D scene model based on the entrance pupil position, the exit pupil position and the distortion map of the physical lens model; Responding to a target light source acquisition condition, calculating a direct contribution value of a target light source to a scattering point of each object surface in each 3D scene model, and updating a plurality of rays of each pixel in each 3D scene model based on the direct contribution value of the target light source to the scattering point of each object surface in each 3D scene model so as to obtain an initial rendering result based on the updated plurality of rays; and generating the dynamic blurred image according to the continuous dynamic pose based on the camera and the initial rendering result.
  4. 4. The method of claim 1, wherein the deriving a camera shake image dataset based on the motion blurred image and the still image comprises: processing the dynamic blurred image and the still image according to a preset image registration method based on the dynamic blurred image and the still image to respectively obtain a registered blurred degraded image and a registered truth image; Obtaining a dynamic blurred image-static image pair based on the registered blurred degraded image and the registered truth image; and cutting the dynamic blurred image-static image pair according to a preset image processing method to obtain a camera shake image data set.
  5. 5. The method of claim 1, wherein the camera's shake trajectory data comprises a plurality of keyframes containing camera 6D pose data.
  6. 6. A camera shake image dataset generation apparatus, comprising: the acquisition module is used for acquiring at least one 3D scene model, the shake track data of the camera and the lens parameters and generating a configuration file corresponding to each rendering flow; The generating module is used for constructing a rendering process based on the configuration file corresponding to each rendering process, processing each 3D scene model according to a preset dynamic camera shake rendering method to obtain a dynamic blurred image, processing each 3D scene model according to a preset still camera rendering method to obtain a still image, and obtaining a camera shake image data set based on the dynamic blurred image and the still image.
  7. 7. The apparatus of claim 6, wherein the obtaining module is specifically configured to: Acquiring scene model information, camera related data and lens parameters corresponding to each scene; Extracting illumination information, background information, 3D scene model parameters and rendering control parameters when each 3D scene model is generated based on scene model information, camera related data and lens parameters corresponding to each scene; classifying illumination information, background information, 3D scene model parameters and rendering control parameters when each 3D scene model is generated according to a preset structure to obtain an image configuration file corresponding to each 3D scene model; and acquiring the space coordinates of the object of each 3D scene model in the rendering configuration file, and determining the surface optical model corresponding to the 3D scene model according to the space coordinates of the object in the image configuration file corresponding to each 3D scene model.
  8. 8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the camera shake image dataset generation method as claimed in any of claims 1-5.
  9. 9. A computer-readable storage medium having stored thereon a computer program, characterized in that the program is executed by a processor for realizing the camera shake image data set generation method according to any one of claims 1 to 5.
  10. 10. A computer program product comprising a computer program which, when executed by a processor, implements a camera shake image data set generation method as claimed in any one of claims 1 to 5.

Description

Camera shake image data set generation method, device, electronic equipment and medium Technical Field The present invention relates to the field of computer vision and graphics, and in particular, to a method, an apparatus, an electronic device, and a medium for generating a camera shake image dataset. Background With the rapid development of deep learning technology, the deep neural network model achieves remarkable effects in computer vision tasks such as image deblurring. However, the performance of these deep neural network models is highly dependent on the quality and authenticity of the training dataset. On one hand, the network needs to learn a key mode of fuzzy degradation from a large number of samples to accurately restore a clear image in actual reasoning, and on the other hand, the distribution of training data must be consistent with the imaging characteristics of a real application scene, otherwise, the model has obvious performance degradation in deployment. Therefore, how to obtain a large-scale, high-fidelity and controllable blurred-sharp image pair has become a key bottleneck restricting the deblurring algorithm from further breaking. In the related art, there are mainly two methods for acquiring a camera shake image dataset, in which (1) a blurred-clear image pair is acquired through real shooting, a shooting scene is manually arranged, a blurred image caused by camera shake in the real scene and a corresponding clear reference image are directly acquired, and in which (2) a blurred image is generated through image processing or simple simulation, such as a convolution kernel simulation method and a multi-frame averaging method. However, the method (1) needs extremely accurate shooting synchronization equipment to ensure effective pairing of the blurred image and the clear image, relies on manual arrangement of shooting scenes, has low scene selection flexibility, has low overall shooting flow efficiency, is difficult to generate data on a large scale, and the convolution kernel simulation method of the method (2) only generates uniform blurring effect, does not consider physical characteristics (such as lens distortion, chromatic dispersion, aberration and the like) of a lens, so that the blurring effect is inconsistent with an image degradation rule of an actual shooting scene, and the multi-frame averaging method can generate unreal blurring stripes, cannot restore a blurring form caused by real camera shake, so that the generated blurring image and the actual shooting image have obvious difference, and have insufficient physical reality and need to be solved. Disclosure of Invention The invention provides a method, a device, electronic equipment and a medium for generating a camera shake image data set, which are used for solving the problems of overlarge gap between a blurred image and an actual image and insufficient physical reality generated in the related technology and improving the reality and physical consistency of the camera shake image data set. In order to achieve the above object, an embodiment of the first aspect of the present invention provides a method for generating a camera shake image dataset, comprising the steps of obtaining at least one 3D scene model, shake locus data of a camera, and lens parameters, generating configuration files corresponding to each rendering flow, constructing a rendering flow based on the configuration files corresponding to each rendering flow, processing each 3D scene model according to a preset dynamic camera shake rendering method to obtain a dynamic blur image, and processing each 3D scene model according to a preset still camera rendering method to obtain a still image, and obtaining a camera shake image dataset based on the dynamic blur image and the still image. Further, in some embodiments, the obtaining at least one scene model, shake trajectory data of a camera, and lens parameters, generating a configuration file corresponding to each rendering process includes obtaining 3D scene model information, camera related data, and lens parameters corresponding to each scene, extracting illumination information, background information, 3D scene model parameters, and rendering control parameters when each 3D scene model is generated based on the scene model information, camera related data, and lens parameters corresponding to each scene, obtaining spatial coordinates of object objects in the image configuration file corresponding to each 3D scene model, and determining a surface optical model corresponding to the 3D scene model according to the spatial coordinates of object objects in the image configuration file corresponding to each 3D scene model. Further, in some embodiments, the steps of constructing a rendering process based on the configuration file corresponding to each rendering process, processing each 3D scene model according to a preset dynamic camera shake rendering method to obtain a dynamic blurred image, processing ea