US-20260127822-A1 - Device-Cloud Collaborative Rendering Method and Related Apparatus

US20260127822A1US 20260127822 A1US20260127822 A1US 20260127822A1US-20260127822-A1

Abstract

An electronic device performs basic rendering on a first device-side scene to obtain first basic rendering data. The electronic device sends a scene identifier of the first device-side scene and a first advanced rendering type to a server. The server stores a first cloud-side scene transformed from the first device-side scene for the first advanced rendering type. The electronic device sends first status data to the server. The server updates the first cloud-side scene based on the first status data, and performs preprocessing of the first advanced rendering type on the first cloud-side scene to obtain first preprocessing data. The server sends the first preprocessing data to the electronic device. The electronic device obtains a first image based on the first basic rendering data and the first preprocessing data, and displays the first image.

Inventors

Baolei Xu
Hongyan Gao
Xiaojie Wang
Wei Wang
Yuanxing Mao

Assignees

HUAWEI TECHNOLOGIES CO., LTD.

Dates

Publication Date: 20260507
Application Date: 20251230
Priority Date: 20230727

Claims (20)

1 . A method, comprising: storing a first device-side scene; performing, on the first device-side scene, basic rendering to obtain first basic rendering data; sending a scene identifier of the first device-side scene and a first advanced rendering type; sending first status data comprising data that is in second status data of the first device-side scene and that is related to preprocessing of the first advanced rendering type; receiving first preprocessing data that are based on preprocessing of the first advanced rendering type on the first cloud-side scene; obtaining, based on the first basic rendering data and the first preprocessing data, a first image having a rendering effect of the first advanced rendering type; and displaying the first image.
2 . The method of claim 1 , further comprising: updating scene information of the first device-side scene to obtain an updated first device-side scene; performing, on the updated first device-side scene, basic rendering to obtain second basic rendering data; determining, based on the second basic rendering data, a second image when a network speed between a first electronic device and a server is lower than a preset value or when a network between the first electronic device and the server is disconnected; and displaying the second image.
3 . The method of claim 1 , wherein sending the scene identifier and the first advanced rendering type comprises sending a first request comprising the scene identifier and the first advanced rendering type, wherein the first request triggers a server to perform preprocessing preparation, wherein before sending the first status data, the method further comprises receiving confirmation information indicating that the preprocessing preparation is completed, and wherein sending the first status data comprises sending, based on the confirmation information, the first status data.
4 . The method of claim 1 , wherein the second status data comprises a part or all of: scene identification information, light source information, role information, camera information, or scene update information.
5 . The method of claim 1 , further comprising obtaining, from an application package of a first application, the scene identifier and the first advanced rendering type.
6 . The method of claim 1 , wherein receiving the first preprocessing data comprises receiving compressed and encoded first preprocessing data, and wherein before obtaining the first image, the method further comprises decoding the compressed and encoded first preprocessing data to obtain decoded first preprocessing data.
7 . The method of claim 1 , wherein the first advanced rendering type is global illumination, wherein the first preprocessing data comprises first irradiance that is of each first pixel in a first imaging picture and that is obtained through global illumination, wherein the first basic rendering data comprises second irradiance of each second pixel in a second imaging picture obtained through basic rendering, wherein third irradiance of a third pixel at a first coordinate in the first image is equal to a product of fourth irradiance of a fourth pixel at the first coordinate in the first preprocessing data and fifth irradiance of a fifth pixel at the first coordinate in the first basic rendering data, and wherein the method further comprises determining, based on the third irradiance, a pixel value of the third pixel.
8 . A method, comprising: receiving a scene identifier of a first device-side scene and a first advanced rendering type; receiving first status data comprising data that is in second status data of the first device-side scene and that is related to preprocessing of the first advanced rendering type; updating, based on the first status data, a first cloud-side scene to obtain an updated first cloud-side scene; performing, on the updated first cloud-side scene, preprocessing of the first advanced rendering type to obtain first preprocessing data; and sending the first preprocessing data to obtain a first image having a lighting effect of the first advanced rendering type.
9 . The method of claim 8 , wherein receiving the scene identifier and the first advanced rendering type comprises receiving a first request comprising the scene identifier and the first advanced rendering type, and wherein before receiving the first status data, the method further comprises: performing, based on the first request, preprocessing preparation comprising: loading the first cloud-side scene; and preparing a rendering pipeline for preprocessing of the first advanced rendering type; and sending confirmation information indicating that the preprocessing preparation is completed.
10 . The method of claim 8 , wherein the second status data comprises a part or all of: scene identification information, light source information, role information, camera information, or scene update information.
11 . The method of claim 8 , further comprising storing a first correspondence among the first device-side scene, the first advanced rendering type, and the first cloud-side scene, and wherein before updating the first cloud-side scene, the method further comprises determining, based on the first correspondence, the scene identifier, and the first advanced rendering type, that a to-be-rendered cloud-side scene is the first cloud-side scene.
12 . The method of claim 8 , further comprising storing a second cloud-side scene transformed from the first device-side scene for a second advanced rendering type.
13 . The method of claim 8 , wherein sending the first preprocessing data comprises sending compressed and encoded first preprocessing data.
14 . The method of claim 8 , wherein the first advanced rendering type is global illumination, wherein the first preprocessing data comprises first irradiance that is of each first pixel in a first imaging picture and that is obtained through global illumination, wherein first basic rendering data comprises second irradiance of each second pixel in a second imaging picture obtained through basic rendering, wherein third irradiance of a third pixel at a first coordinate in the first image is equal to a product of fourth irradiance of a fourth pixel at the first coordinate in the first preprocessing data and fifth irradiance of a fifth pixel at the first coordinate in the first basic rendering data, and wherein the method further comprises determining, based on the third irradiance, a pixel value of the third pixel.
15 . The method of claim 8 , further comprising sending the first preprocessing data when the first preprocessing data is unrelated to a field of view.
16 . An electronic device, comprising: a memory configured to store instructions; and one or more processors coupled to the memory and configured to execute the instructions to cause the electronic device to: store a first device-side scene; perform, on the first device-side scene, basic rendering to obtain first basic rendering data; send, to a server, a scene identifier of the first device-side scene and a first advanced rendering type; send, to the server, first status data comprising data that is in second status data of the first device-side scene and that is related to preprocessing of the first advanced rendering type; receive first preprocessing data that are based on preprocessing of the first advanced rendering type on the first cloud-side scene; obtain, based on the first basic rendering data and the first preprocessing data, a first image having a rendering effect of the first advanced rendering type; and display the first image.
17 . The electronic device of claim 16 , wherein the one or more processors are further configured to execute the instructions to cause the electronic device to: update scene information of the first device-side scene to obtain an updated first device-side scene; perform, on the updated first device-side scene, basic rendering to obtain second basic rendering data; determine, based on the second basic rendering data, a second image when a network speed between the electronic device and the server is lower than a preset value, or when a network between the electronic device and the server is disconnected; and display the second image.
18 . The electronic device of claim 16 , wherein the one or more processors are further configured to execute the instructions to cause the electronic device to further send the scene identifier and the first advanced rendering type by sending a first request comprising the scene identifier and the first advanced rendering type, wherein the first request triggers the server to perform preprocessing preparation, wherein before sending the first status data, the one or more processors are further configured to execute the instructions to receive confirmation information indicating that the preprocessing preparation is completed, and wherein the one or more processors are further configured to execute the instructions to cause the electronic device to send the first status data by sending, based on the confirmation information, the first status data.
19 . The electronic device of claim 16 , wherein the second status data comprises a part or all of: scene identification information, light source information, role information, camera information, or scene update information.
20 . The electronic device of claim 16 , wherein the one or more processors are further configured to execute the instructions to obtain, from an application package of a first application, the scene identifier and the first advanced rendering type.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This is a continuation of International Patent Application No. PCT/CN2024/107527 filed on Jul. 25, 2024, which claims priority to Chinese Patent Application No. 202310940383.X filed on Jul. 27, 2023. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties. TECHNICAL FIELD This application relates to the field of electronic technologies, and in particular, to a device-cloud collaborative rendering method and a related apparatus. BACKGROUND Three-dimensional (3D) applications (such as 3D games and 3D modeling software) on terminal devices such as mobile phones are increasing. Providing a rendering capability for the 3D applications is a core function of the terminal devices for supporting running of the 3D applications. However, a graphics processing unit (GPU) (for ease of description, referred to as a lightweight GPU) of a terminal device (for example, a mobile device) whose size and power consumption are limited has a considerable gap compared with a personal computer (PC)-level GPU. For example, single-precision computing power of the Apple A15 in 2021 was 1.5 tera floating point operations per second (TFLOPS), but single-precision computing power of the NVIDIA GTX 580 in 2010 has reached 1.58 TFLOPS, and computing power of the NVIDIA RTX 3080Ti in 2021 has reached 27.955 TFLOPS. It can be learned that computing power development of the lightweight GPU on the mobile device lags behind the PC-level GPU by about ten years. A GPU on a cloud server (namely, the PC-level GPU) has a huge computing power advantage over the lightweight GPU on the mobile device. With development of cloud computing technologies, transferring computing power of a device side to a cloud side to collaborate with the cloud side to process a service on the device side is gradually becoming a technology evolution direction. Currently, how to collaborate with the cloud side to implement efficient rendering processing, to achieve rendering effect of a ray tracing technology on the device side remains to be studied. SUMMARY This application provides a device-cloud collaborative rendering method and a related apparatus, to reduce a computing power requirement for and load of a device-side GPU, so that an electronic device with low GPU computing power can also present advanced rendering effect, for example, rendering effect of a ray tracing technology. This effectively improves user experience. According to a first aspect, this application provides a device-cloud collaborative rendering method, applied to a communication system. The communication system includes a first electronic device and a server. The first electronic device stores at least one device-side scene of a first application, and a first device-side scene is any one of the at least one device-side scene. The server stores a first cloud-side scene transformed from the first device-side scene for a first advanced rendering type. The method includes: the first electronic device performs basic rendering on the first device-side scene to obtain first basic rendering data; the first electronic device sends a scene identifier of the first device-side scene and the first advanced rendering type to the server; the first electronic device sends first status data to the server, where the first status data includes data that is in status data of the first device-side scene and that is related to preprocessing of the first advanced rendering type; the server updates the first cloud-side scene based on the first status data, and performs preprocessing of the first advanced rendering type on an updated first cloud-side scene, to obtain first preprocessing data; the server sends the first preprocessing data to the first electronic device; the first electronic device obtains a first image based on the first basic rendering data and the first preprocessing data, where the first image has lighting effect of the first advanced rendering type; and the first electronic device displays the first image. During implementation of this embodiment of this application, one device-side scene is transformed into one cloud-side scene for each advanced rendering type, and the server stores the cloud-side scene. For a specific advanced rendering type, the electronic device needs to have only a capability of performing basic rendering on a device-side scene, and the server implements preprocessing of advanced rendering on a cloud-side scene corresponding to the device-side scene. After basic rendering data is fused with the preprocessing data of advanced rendering, a finally rendered image presents lighting effect of the specific advanced rendering type. In this way, for various advanced rendering types, a computing power requirement for and load of a device-side GPU can be reduced. The electronic device (for example, a mobile phone or a tablet) can present advanced rendering effect without requiring high-standard hardware