US-12626327-B2 - Image transmission method and device, image display method and device, and system

US12626327B2US 12626327 B2US12626327 B2US 12626327B2US-12626327-B2

Abstract

An image transmission method comprises obtaining a spherical panoramic image based on a wide-angle image acquired from a target environment; obtaining an image of interest by cropping the spherical panoramic image based on a preset field of view and an Euler angle corresponding to a rotation attitude of a target object when using a virtual reality device; stitching the image of interest and a down-sampled wide-angle image to obtain a stitched image; the down-sampled wide-angle image being obtained by down-sampling the wide-angle image; and transmitting the stitched image to the virtual reality device.

Inventors

Dahao LIN
Xiaoshen Tian

Assignees

ARASHI VISION INC.

Dates

Publication Date: 20260512
Application Date: 20250310
Priority Date: 20220913

Claims (19)

1 . An image transmission method, comprising: obtaining a spherical panoramic image based on a wide-angle image acquired from a target environment; obtaining an image of interest by cropping the spherical panoramic image based on a preset field of view and an Euler angle corresponding to a rotation attitude of a target object when using a virtual reality device; stitching the image of interest and a down-sampled wide-angle image to obtain a stitched image, the down-sampled wide-angle image being obtained by down-sampling the wide-angle image; and transmitting the stitched image to the virtual reality device.
2 . The method according to claim 1 , wherein the obtaining the spherical panoramic image based on the wide-angle image acquired from the target environment comprises: acquiring the wide-angle image of the target environment by an image acquirer capturing the target environment; generating a panoramic image based on the wide-angle image; and rendering the panoramic image according to a spherical model to obtain the spherical panoramic image.
3 . The method according to claim 2 , wherein the acquiring the wide-angle image of the target environment by the image acquirer capturing the target environment and generating the panoramic image based on the wide-angle image comprises: using two fisheye lenses mounted on an unmanned aerial vehicle (UAV) as the image acquirer to shoot an aerial environment of the UAV to obtain two wide-angle images; and stitching the two wide-angle images to obtain the panoramic image.
4 . The method according to claim 2 , wherein the acquiring the wide-angle image of the target environment by the image acquirer capturing the target environment and generating the panoramic image based on the wide-angle image comprises: using two wide-angle lenses on the image acquirer to capture videos of the target environment; extracting video frames sequentially according to frame sequence numbers from each of the captured videos of the target environment; stitching video frames having the same sequence number from the extracted video frames to obtain the panoramic image.
5 . The method according to claim 1 , wherein the down-sampled wide-angle image comprises a first image and a second image; and the stitching the image of interest and the down-sampled wide-angle image to obtain the stitched image comprises: stitching the first image and the second image in an up-down direction to obtain a third image; and stitching the third image and the image of interest in a left-right direction to obtain the stitched image.
6 . The method according to claim 5 , wherein the first image is a wide-angle image of a left eye perspective, and the second image is another wide-angle image of a right eye perspective.
7 . The method according to claim 1 , wherein the obtaining the image of interest by cropping the spherical panoramic image based on the preset field of view and the Euler angle corresponding to the rotation attitude of the target object when using the virtual reality device comprises: taking a center of a spherical model as an observation point, determining a region of interest based on the preset field of view; rotating the region of interest according to the Euler angle corresponding to the rotation attitude of the target object when using the virtual reality device to obtain a rotated region of interest; and obtaining the image of interest by cropping the spherical panoramic image based on the rotated region of interest.
8 . The method according to claim 7 , wherein the image of interest is an image within an effective field of view of the target object.
9 . The method according to claim 1 , wherein the stitching the image of interest with the down-sampled wide-angle image to obtain the stitched image comprises: determining a sampling rate corresponding to the wide-angle image based on an image resolution of the image of interest; down-sampling the wide-angle image according to the sampling rate to obtain the down-sampled wide-angle image; and stitching the image of interest and the down-sampled wide-angle image to obtain the stitched image according to a preset image arrangement mode.
10 . The method according to claim 9 , wherein the image of interest does not overlap with the down-sampled wide-angle image in the stitched image.
11 . The method according to claim 1 , wherein a resolution of the down-sampled wide-angle image is smaller than that of the image of interest.
12 . An image display method, comprising: receiving a stitched image transmitted from an image acquirer, the stitched image comprising an image of interest and a down-sampled wide-angle image; obtaining the image of interest and the down-sampled wide-angle image by cropping the stitched image; generating a down-sampled panoramic image based on the down-sampled wide-angle image; rendering the down-sampled panoramic image according to a spherical model to obtain a down-sampled spherical panoramic image; synthesizing the down-sampled spherical panoramic image and the image of interest to obtain a synthesized image based on a preset field of view and an Euler angle corresponding to the stitched image; and displaying the synthesized image on a screen.
13 . The method according to claim 12 , wherein a resolution of the down-sampled wide-angle image is smaller than that of the image of interest.
14 . The method according to claim 12 , wherein the synthesizing the down-sampled spherical panoramic image and the image of interest to obtain the synthesized image based on the preset field of view and the Euler angle corresponding to the stitched image comprises: taking a center of a spherical model as an observation point; generating a plane model based on the preset field of view; rendering the image of interest on the plane model to obtain a planar image; rotating the planar image according to the Euler angle corresponding to the stitched image, and superimposing the rotated planar image on the down-sampled spherical panoramic image.
15 . The method according to claim 14 , wherein the superimposing the rotated planar image on the down-sampled spherical panoramic image comprises: using the rotated planar image as a planar foreground and the down-sampled spherical panoramic image as a spherical background; and superimposing the planar foreground and the spherical background.
16 . The method according to claim 12 , wherein the screen is on a virtual reality device, and the Euler angle is an angle representing a rotation attitude of a target object when using the virtual reality device.
17 . The method according to claim 12 , wherein the image of interest is displayed in a region of interest of the target object and the down-sampled spherical panoramic image is displayed in another region other than the region of interest of the target object.
18 . A system, comprising an image transmission device; wherein, the image transmission device is configured to: generate an image of interest based on a wide-angle image obtained by capturing a target environment, based on a preset field of view and rotation information of a virtual reality device; stitch the image of interest with a down-sampled environment image to obtain a stitched image, the down-sampled environment image obtained by down-sampling the wide-angle image; and transmit the stitched image to the virtual reality device, and wherein the system further comprises the virtual reality device, and the virtual display device is configured to: obtain the stitched image, the stitched image comprising: the image of interest and the down-sampled environment image; based on rotation information corresponding to the stitched image, synthesize the down-sampled environment image and the image of interest; and control a screen to display the synthesized image.
19 . The system of claim 18 , wherein the down-sampled environment image comprises a panoramic image obtained by stitching a plurality of down-sampled wide-angle images.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS The present application is a continuation of International Application No. PCT/CN2023/117974, filed Sep. 11, 2023, which claims priority of Chinese Patent Application No. 202211108814.8, filed Sep. 13, 2022, the entire content of both of which being incorporated herein by reference in its entirety. TECHNICAL OF FIELD The present application relates to a field of computer technology, and in particular to an image transmission method and apparatus, an image display method and apparatus, a computer device, a storage medium, and a computer program product. BACKGROUND With the development of computer technology, application of virtual reality technology is becoming more and more widespread. In a virtual reality mode, in order for a user to watch high-definition images, it is necessary to transmit panoramic resources with a large amount of data to a virtual reality device, which requires a large transmission bandwidth. SUMMARY Based on this, it is necessary to provide an image transmission method, image display method, and a system that can reduce the transmission bandwidth in order to solve the above and other technical problems. In a first aspect, the present application provides an image transmission method. The method comprises obtaining a spherical panoramic image based on a wide-angle image acquired from a target environment; obtaining an image of interest by cropping the spherical panoramic image based on a preset field of view and an Euler angle corresponding to a rotation attitude of a target object when using a virtual reality device; stitching the image of interest and a down-sampled wide-angle image to obtain a stitched image; the down-sampled wide-angle image being obtained by down-sampling the wide-angle image; and transmitting the stitched image to the virtual reality device. In a second aspect, the present application provides an image display method. The method comprises: receiving a stitched image transmitted from an image acquirer, the stitched image comprising an image of interest and a down-sampled wide-angle image; obtaining the image of interest and the down-sampled wide-angle image by cropping the stitched image; generating a down-sampled panoramic image based on the down-sampled wide-angle image; rendering the down-sampled panoramic image according to a spherical model to obtain a down-sampled spherical panoramic image; synthesizing the down-sampled spherical panoramic image and the image of interest to obtain a synthesized image based on a preset field of view and an Euler angle corresponding to the stitched image; and displaying the synthesized image on a screen. In a third aspect, the present application provides a system. The system includes an image acquirer, wherein the image acquirer comprises first circuitry configured to: obtain a spherical panoramic image based on a wide-angle image acquired from a target environment; obtain an image of interest by cropping the spherical panoramic image based on a preset field of view and an Euler angle corresponding to a rotation attitude of a target object when using a virtual reality device; stitch the image of interest and a down-sampled wide-angle image to obtain a stitched image; the down-sampled wide-angle image being obtained by down-sampling the wide-angle image; and transmit the stitched image to the virtual reality device. In a fourth aspect, the system may further include an image display, wherein the image display comprises second circuitry configured to receive the stitched image transmit from the image acquirer; obtain the image of interest and the down-sampled wide-angle image by cropping the stitched image; generate a down-sampled panoramic image based on the down-sampled wide-angle image; render the down-sampled panoramic image according to a spherical model to obtain a down-sampled spherical panoramic image; synthesize the down-sampled spherical panoramic image and the image of interest to obtain a synthesized image based on the preset field of view and the Euler angle corresponding to the stitched image; and display the synthesized image on a screen. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the accompanying drawings to be used in the embodiments will be briefly introduced below, and it will be obvious that the accompanying drawings in the following description are only some of the embodiments of the present disclosure, and that for the person of ordinary skill in the field, other accompanying drawings can be obtained based on these drawings, without giving creative labor. FIG. 1 is a diagram showing an application environment of an image transmission method according to one embodiment; FIG. 2 is a diagram showing an image transmission method according to one embodiment; FIG. 3 is a schematic diagram of an image arrangement mode according to one embodiment; FIG. 4 is a schematic diagram of