Search

CN-122002121-A - Image processing method, electronic device, chip system and readable storage medium

CN122002121ACN 122002121 ACN122002121 ACN 122002121ACN-122002121-A

Abstract

The embodiment of the application provides an image processing method, electronic equipment, a chip system and a readable storage medium, wherein the method can comprise the steps of shooting a first image and a second image with different visual angles according to a target scene, and determining camera internal and external parameter information and scene point cloud information; the method comprises the steps of determining first attribute information of a scene Gaussian ball based on camera internal and external parameters and scene point cloud information, determining a target depth image of the scene Gaussian ball according to the first attribute information by referring to a shooting view angle and a reference light center position, obtaining second attribute information of the scene Gaussian ball based on the target depth image and color truth value information, generating a target image with the shooting view angle being the reference shooting view angle under the target scene based on the reference shooting view angle and the second attribute information, wherein the first depth image comprises a depth value of an intersection point of an observation ray and the scene Gaussian ball from the reference light center position, and an end point position of the observation ray is the reference light center position. In this way, the quality of the new view angle image synthesis can be improved.

Inventors

  • YI WENTAO

Assignees

  • 荣耀终端股份有限公司

Dates

Publication Date
20260508
Application Date
20241107

Claims (11)

  1. 1. An image processing method, the method comprising: Determining camera internal and external parameter information and scene point cloud information in a target scene according to a first image and a second image in the target scene, wherein the shooting visual angles of the first image and the second image are different; Determining first attribute information of a scene Gaussian ball based on the camera internal and external parameters and the scene point cloud information; According to the first attribute information, referring to a shooting visual angle and a reference optical center position, determining a target depth image of the scene Gaussian sphere; the target depth map comprises depth values of the intersection point of the observation ray and the scene Gaussian ball from the reference optical center position, wherein the end point position of the observation ray is the reference optical center position, and the direction of the observation ray is in the view angle range of the reference shooting view angle; based on the target depth image and the color truth value information, the first attribute information is adjusted to obtain second attribute information of the Gaussian ball of the scene; And generating a target image in the target scene based on the reference shooting view angle and the second attribute information, wherein the shooting view angle of the target image is the reference shooting view angle.
  2. 2. The method of claim 1, wherein determining the target depth image of the scene gaussian sphere based on the first attribute information, the reference shooting angle of view, and the reference optical center position, comprises: The first attribute information refers to a shooting visual angle, a reference optical center position and a depth priori image, and a first depth image of the scene Gaussian ball is determined; performing edge extraction processing on the first depth image, and determining a non-edge area of the first depth image; and carrying out depth regularization constraint processing on the non-edge region of the first depth image to obtain a target depth image.
  3. 3. The method of claim 2, wherein the determining a first depth image of the scene gaussian ball based on the first attribute information, reference shooting perspective, reference optical center position, and depth prior image comprises: inputting the first attribute information and the reference shooting visual angle into a multi-layer sensing network to obtain a first distance value output by the multi-layer sensing network, wherein the first distance value is a distance value between the intersection point and the reference optical center position; determining first position information based on the reference optical center position and the first distance value, wherein the first position information is the position information of an intersection point of an observation ray and the scene Gaussian ball; Determining a second distance value based on the first position information and the first distance value, wherein the second distance value is a distance value between a projection point of the intersection point in a camera direction and the reference optical center position; Determining a first depth value of the scene gaussian sphere based on the second distance value; a first depth image of the scene gaussian sphere is determined based on the first depth value.
  4. 4. A method according to claim 2 or 3, characterized in that the method further comprises: performing monocular depth estimation on a monocular image, and determining a first monocular depth image corresponding to the monocular image, wherein the monocular image is the first image or the second image; performing physical scale registration processing on the first monocular depth image based on the reference sparse depth image to obtain a second monocular depth image; And carrying out fusion processing on the reference sparse depth map and the second monocular depth image to obtain the depth prior image.
  5. 5. The method according to claim 4, wherein the method further comprises: Determining monocular scene point cloud information of the monocular image; And determining the reference sparse depth image based on the monocular scene point cloud information of the monocular image.
  6. 6. The method of any one of claims 1-5, wherein the first attribute information comprises first geometric attribute information and first color attribute information; The adjusting the first attribute information based on the target depth image and the color truth value information to obtain second attribute information of the scene Gaussian ball includes: Based on the target depth image, adjusting the first geometric attribute information to obtain second geometric attribute information; adjusting the first color attribute information based on the color truth value information to obtain second color attribute information; And determining second attribute information of the Gaussian ball of the scene, wherein the second attribute information comprises the second geometric attribute information and the second color attribute information.
  7. 7. The method of claim 6, wherein the first geometric attribute information comprises one or more of size information, orientation information, center of sphere position information, opacity, and wherein the first color attribute information comprises color information.
  8. 8. The method according to any one of claims 1-7, further comprising: The color truth information is determined based on color information of the first image and/or the second image.
  9. 9. An electronic device comprising a memory, one or more processors, a plurality of application programs, and one or more programs, wherein the one or more programs are stored in the memory, wherein the one or more processors, when executing the one or more programs, cause the electronic device to implement the method of any of claims 1-8.
  10. 10. A chip system comprising at least one processor, a memory and an interface circuit, the memory, the interface circuit and the at least one processor being interconnected by a circuit, the at least one memory having program instructions stored therein, which when executed by the processor, cause the chip system to perform the method of any of claims 1-8.
  11. 11. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method according to any of claims 1-8.

Description

Image processing method, electronic device, chip system and readable storage medium Technical Field The present application relates to the field of image processing technologies, and in particular, to an image processing method, an electronic device, a chip system, and a readable storage medium. Background New view synthesis refers to obtaining geometric representation and color attribute of a scene based on a plurality of view images in the same scene, so as to generate a new view image in the same scene. New view angle synthesis is widely used in a large number of fields such as augmented reality (augmented reality, AR), virtual Reality (VR), autopilot, and three-dimensional content generation. Common new view angle synthesis methods can be applied to a wide range of scenes, but the quality of the generated new view angle images is highly dependent on the accuracy of scene geometry information (e.g. object position, shape, size in the scene) and information of dense known view angle images. Therefore, in a limited scene such as sparse information of known view images (e.g., a small number of known view images), unknown parameters inside and outside a camera, unknown view sequence of view images, etc., the quality of the new view images synthesized by the method is poor. Disclosure of Invention The embodiment of the application provides an image processing method, electronic equipment, a chip system and a readable storage medium, which are beneficial to improving the quality of new view angle image synthesis. According to a first aspect, an embodiment of the application provides an image processing method, which comprises the steps of determining camera inside-outside parameter information and scene point cloud information in a target scene according to a first image and a second image in the target scene, wherein the first image is different from the second image in a shooting view angle, determining first attribute information of a scene Gaussian ball based on the camera inside-outside parameter and the scene point cloud information, determining a target depth image of the scene Gaussian ball according to the first attribute information, a reference shooting view angle and a reference light center position, determining a shooting view angle different from the shooting view angles of the first image and the second image, wherein the target depth image comprises a depth value of an intersection point of an observation ray and the scene Gaussian ball from the reference light center position, the end point position of the observation ray is the reference light center position, the direction of the observation ray is in the view angle range of the reference shooting view angle, adjusting the first attribute information based on the target depth image and the color truth value information, and obtaining second attribute information of the scene Gaussian ball, and generating the target image in the target scene based on the reference shooting view angle and the second attribute information, wherein the shooting view angle of the target image in the target scene is the reference shooting view angle. According to the application, under the condition that the internal and external parameters of the camera are unknown and the visual angle sequence is unknown, a new visual angle image, namely a target image, in the target scene can be generated based on two images with different shooting visual angles, namely a first image and a second image, in the target scene. And the first attribute information of the scene Gaussian sphere can be optimized and adjusted based on the target depth image and the color truth value information to obtain the second attribute information, so that the generation of a target image which is closer to a real target scene is facilitated, and the image quality of the target image is improved. In one possible implementation, determining the target depth image of the Gaussian sphere of the scene according to the first attribute information, the reference shooting view angle and the reference optical center position can comprise determining the first depth image of the Gaussian sphere of the scene based on scene point cloud information, the first attribute information, the reference shooting view angle, the reference optical center position and the depth prior image, performing edge extraction processing on the first depth image to determine a non-edge area of the first depth image, and performing depth regularization constraint processing on the non-edge area of the first depth image to obtain the target depth image. In this way, the accuracy of the first depth image may be improved. And the target depth image with smoothly changed depth values can be obtained by carrying out depth regularization constraint processing on the non-edge region of the first depth image, which is favorable for improving the accuracy of the depth values of the target depth image and enabling the Gaussian balls of the