US-12620124-B2 - Image processing method and apparatus, device, storage medium, and computer program product

Abstract

Embodiments of this application provide an image processing method performed by a computer device. The method includes: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; and fusing a target object image and a virtual background image in the actual image based on intrinsic information and the pose information to obtain a target image.

Inventors

• Fasheng Chen
• Zhiyang LIN
• Lei Sun

Assignees

• TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

Dates

Publication Date

20260505

Application Date

20230607

Priority Date

20211124

Claims (17)

1. 1 . An image processing method, performed by a computer device, the method comprising: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; determining a field of view of the camera based on the intrinsic information; determining attribute information of a virtual photographing device in a virtual world through a rendering engine based on the intrinsic information, the target pose information of the camera, and the field of view; determining target position information of a target imaging plane of the target object image in the virtual world based on the attribute information of the virtual photographing device and a reference position point in the target object image; and fusing the target object image and a virtual background image based on the target position information of the target imaging plane to obtain a target image.
2. 2 . The method according to claim 1 , further comprising: in response to the actual image not comprising a position reference pattern, obtaining historical pose information of the camera in the world coordinate system when the camera photographs a previous actual image as the target pose information.
3. 3 . The method according to claim 1 , wherein the determining first position information of the position reference pattern in an image coordinate system comprises: obtaining at least one feature point contained in the position reference pattern and a first feature point coordinate of each feature point in the image coordinate system; and taking the obtained at least one feature point and the first feature point coordinate as the first position information of the position reference pattern.
4. 4 . The method according to claim 1 , wherein the determining second position information of the position reference pattern in a world coordinate system comprises: obtaining a pattern number embedded in the position reference pattern; querying a feature point number of each feature point and a second feature point coordinate of each feature point in the world coordinate system based on the pattern number; and taking the queried feature point number of each feature point and the second feature point coordinate of each feature point as the second position information of the position reference pattern.
5. 5 . The method according to claim 1 , wherein the actual image further comprises a background image, and the background image comprises a green screen region and a non-green screen region; and before the fusing the target object image and a virtual background image based on the target position information of the target imaging plane to obtain the target image, the method further comprises: obtaining third position information of a boundary line between the green screen region and the non-green screen region in the world coordinate system; determining fourth position information of the boundary line in the image coordinate system based on the third position information, the intrinsic information, and the target pose information; removing an image of the green screen region in the actual image to obtain a pre-processed image; and removing, based on the fourth position information, an image of the non-green screen region located outside the boundary line in the pre-processed image to obtain the target object image.
6. 6 . The method according to claim 1 , wherein the fusing the target object image and the virtual background image based on the target position information of the target imaging plane to obtain the target image comprises: determining target size information of the target imaging plane based on the attribute information of the virtual photographing device and the target position information of the target imaging plane; and fusing the target object image and the virtual background image based on the target position information and the target size information to obtain the target image.
7. 7 . A computer device, comprising a memory and a processor, the memory being configured to store computer programs executed on the processor; and the processor being configured to, when executing the computer programs, cause the computer device to perform an image processing method including: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; determining a field of view of the camera based on the intrinsic information; determining attribute information of a virtual photographing device in a virtual world through a rendering engine based on the intrinsic information, the target pose information of the camera, and the field of view; determining target position information of a target imaging plane of the target object image in the virtual world based on the attribute information of the virtual photographing device and a reference position point in the target object image; and fusing the target object image and a virtual background image based on the target position information of the target imaging plane to obtain a target image.
8. 8 . The computer device according to claim 7 , wherein the method further comprises: in response to the actual image not comprising a position reference pattern, obtaining historical pose information of the camera in the world coordinate system when the camera photographs a previous actual image as the target pose information.
9. 9 . The computer device according to claim 7 , wherein the determining first position information of the position reference pattern in an image coordinate system comprises: obtaining at least one feature point contained in the position reference pattern and a first feature point coordinate of each feature point in the image coordinate system; and taking the obtained at least one feature point and the first feature point coordinate as the first position information of the position reference pattern.
10. 10 . The computer device according to claim 7 , wherein the determining second position information of the position reference pattern in a world coordinate system comprises: obtaining a pattern number embedded in the position reference pattern; querying a feature point number of each feature point and a second feature point coordinate of each feature point in the world coordinate system based on the pattern number; and taking the queried feature point number of each feature point and the second feature point coordinate of each feature point as the second position information of the position reference pattern.
11. 11 . The computer device according to claim 7 , wherein the actual image further comprises a background image, and the background image comprises a green screen region and a non-green screen region; and before the fusing the target object image and a virtual background image based on the target position information of the target imaging plane to obtain the target image, the method further comprises: obtaining third position information of a boundary line between the green screen region and the non-green screen region in the world coordinate system; determining fourth position information of the boundary line in the image coordinate system based on the third position information, the intrinsic information, and the target pose information; removing an image of the green screen region in the actual image to obtain a pre-processed image; and removing, based on the fourth position information, an image of the non-green screen region located outside the boundary line in the pre-processed image to obtain the target object image.
12. 12 . The computer device according to claim 7 , wherein the fusing the target object image and the virtual background image based on the target position information of the target imaging plane to obtain the target image comprises: determining target size information of the target imaging plane based on the attribute information of the virtual photographing device and the target position information of the target imaging plane; and fusing the target object image and the virtual background image based on the target position information and the target size information to obtain the target image.
13. 13 . A non-transitory computer-readable storage medium, storing computer programs that, when executed by a processor of a computer device, cause the computer device to perform an image processing method including: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; determining a field of view of the camera based on the intrinsic information; determining attribute information of a virtual photographing device in a virtual world through a rendering engine based on the intrinsic information, the target pose information of the camera, and the field of view; determining target position information of a target imaging plane of the target object image in the virtual world based on the attribute information of the virtual photographing device and a reference position point in the target object image; and fusing the target object image and a virtual background image based on the target position information of the target imaging plane to obtain a target image.
14. 14 . The non-transitory computer-readable storage medium according to claim 13 , wherein the method further comprises: in response to the actual image not comprising a position reference pattern, obtaining historical pose information of the camera in the world coordinate system when the camera photographs a previous actual image as the target pose information.
15. 15 . The non-transitory computer-readable storage medium according to claim 13 , wherein the determining first position information of the position reference pattern in an image coordinate system comprises: obtaining at least one feature point contained in the position reference pattern and a first feature point coordinate of each feature point in the image coordinate system; and taking the obtained at least one feature point and the first feature point coordinate as the first position information of the position reference pattern.
16. 16 . The non-transitory computer-readable storage medium according to claim 13 , wherein the determining second position information of the position reference pattern in a world coordinate system comprises: obtaining a pattern number embedded in the position reference pattern; querying a feature point number of each feature point and a second feature point coordinate of each feature point in the world coordinate system based on the pattern number; and taking the queried feature point number of each feature point and the second feature point coordinate of each feature point as the second position information of the position reference pattern.
17. 17 . The non-transitory computer-readable storage medium according to claim 13 , wherein the actual image further comprises a background image, and the background image comprises a green screen region and a non-green screen region; and before the fusing the target object image and a virtual background image based on the target position information of the target imaging plane to obtain the target image, the method further comprises: obtaining third position information of a boundary line between the green screen region and the non-green screen region in the world coordinate system; determining fourth position information of the boundary line in the image coordinate system based on the third position information, the intrinsic information, and the target pose information; removing an image of the green screen region in the actual image to obtain a pre-processed image; and removing, based on the fourth position information, an image of the non-green screen region located outside the boundary line in the pre-processed image to obtain the target object image.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation application of PCT Patent Application No. PCT/CN2022/121602, entitled “IMAGE PROCESSING METHOD AND APPARATUS, DEVICE, STORAGE MEDIUM, AND COMPUTER PROGRAM PRODUCT” filed on Sep. 27, 2022, which claims priority to Chinese Patent Application No. 202111406521.3, entitled “IMAGE PROCESSING METHOD AND APPARATUS, DEVICE, STORAGE MEDIUM, AND COMPUTER PROGRAM PRODUCT” filed on Nov. 24, 2021, all of which is incorporated by reference in its entirety. FIELD OF THE TECHNOLOGY Embodiments of this application relate to the field of computer technologies, and in particular, to an image processing method and apparatus, a device, a storage medium, and a computer program product. BACKGROUND OF THE DISCLOSURE A virtual film production technology is to introduce a real object into a virtual scene, and establish an interactive feedback information loop among a virtual world, a real world, and the real object, so as to enhance the sense of reality. The mixed reality technology between reality and virtual has become a frontier practical technology in the fields of exhibition, experience, training, medical treatment, historical site restoration, industrial maintenance, and the like. In the related art, during virtual film production, a locator is installed on a photographing device to obtain pose information of the photographing device, the pose information is used for controlling rendering of a virtual scene, and then a subject (person, item, or the like) in a real scene photographed by the photographing device is superimposed on the virtual scene. However, the virtual scene obtained by the foregoing method often does not match with a real subject, thereby causing rendering errors of a picture, and thus resulting in a poor presentation effect of a video obtained by the virtual film production. SUMMARY Embodiments of this application provide an image processing method and apparatus, a computer device, a computer storage medium, and a computer program product, which can improve the accuracy of picture rendering and the rendering effect of a video obtained by virtual film production. An embodiment of this application provides an image processing method performed by a computer device. The method includes: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern;determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively;determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; andfusing the target object image and a virtual background image based on the intrinsic information and the target pose information to obtain a target image. An embodiment of this application provides a computer device, including a memory and a processor. The memory is configured to store computer programs executed on the processor. The processor is configured to implement, when executing the computer programs, the image processing method provided in the foregoing embodiment of this application. An embodiment of this application provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores computer programs executable by a computer device. When the computer programs are executed on the computer device, the computer device performs the image processing method provided in the foregoing embodiment of this application. The embodiments of this application have the following beneficial effects. In the embodiments of this application, after a actual image photographed by a camera is obtained, target pose information of the camera is determined according to first position information of a position reference pattern in the actual image in an image coordinate system and second position information in a world coordinate system. In this way, since the target pose information of the camera is determined according to the position information of the position reference pattern in different coordinate systems in the image photographed by the device, the photographing of the actual image and the determination of the target pose information are synchronously aligned. Then, a target object image in the actual image and a virtual background image are fused based on intrinsic information and the target pose information, thereby ensuring the accuracy of picture rendering, and improving a presentation effect of a video obtained by virtual film production. Furthermore, in the embodiments of this application, pose information can be obtained without a hardware locator, and the problem of synchronization between the hardware locator and a photographing picture can be solved without professional-level camera equ