EP-4742166-A1 - METHODS FOR RENDERING AN IMAGE OF A THREE-DIMENSIONAL SCENE

Abstract

A method for rendering an image of a three-dimensional scene using path tracing. Scene feature data for a pixel of the image to be rendered using path tracing is obtained, indicating visual features of a location of the three-dimensional scene for depiction by the pixel in the image, is obtained. The scene feature data is processed at an artificial neural network, ANN, to determine a budget allocation parameter for rendering the pixel using path tracing. The budget allocation parameter is indicative of an amount of computing resources to be used for rendering the pixel using path tracing. The ANN is trained to determine, from the scene feature data, the budget allocation parameter based on the visual features indicated by the scene feature data. The determined budget allocation parameter is output from the ANN to control a rendering of the pixel using path tracing.

Inventors

• TREDER, Matthias
• Lutz, Sebastian

Assignees

• Sony Interactive Entertainment Europe Limited

Dates

Publication Date

20260513

Application Date

20241107

Claims (15)

1. A computer-implemented method for rendering an image of a three-dimensional scene using path tracing, the method comprising, for a pixel of the image to be rendered using path tracing: obtaining scene feature data for the pixel, the scene feature data indicating visual features of a location of the three-dimensional scene for depiction by the pixel in the image; processing, at an artificial neural network, ANN, the scene feature data to determine a budget allocation parameter for rendering the pixel using path tracing, the budget allocation parameter indicative of an amount of computing resources to be used for rendering the pixel using path tracing, wherein the ANN is trained to determine, from the scene feature data, the budget allocation parameter based on the visual features indicated by the scene feature data; and outputting, from the ANN, the determined budget allocation parameter to control a rendering of the pixel using path tracing.
2. A computer-implemented method according to claim 1, the method comprising rendering the pixel by performing path tracing using the determined budget allocation parameter.
3. A computer-implemented method according to claim 2, the method comprising generating the image of the three-dimensional scene using the rendered pixel.
4. A computer-implemented method according to any preceding claim, wherein the budget allocation parameter is indicative of a number of light paths to be traced for rendering the pixel using path tracing.
5. A computer-implemented method according to any preceding claim, wherein the budget allocation parameter is indicative of a maximum path length of light paths to be traced for rendering the pixel using path tracing.
6. A computer-implemented method according to any preceding claim, the method comprising: obtaining a system resource characteristic of a system configured to render the image using path tracing; and using the system resource characteristic and the determined budget allocation parameter to control the rendering, by the system, of the pixel using path tracing.
7. A computer-implemented method according to claim 5 or claim 6, wherein the system resource characteristic is indicative of a total number of light paths to be traced for rendering the image.
8. A computer-implemented method according to any preceding claim, wherein the visual features comprise one or more of: geometric features indicating a geometry of one or more objects and/or surfaces in the scene; and material features indicating physical and/or optical properties of one or more objects and/or surfaces in the scene.
9. A computer-implemented method according to any preceding claim, wherein the scene feature data is indicative of a number, location and/or type of light sources in the scene.
10. A computer-implemented method according to any preceding claim, wherein the scene feature data for the pixel is derived using a ray tracing process, and wherein the visual features indicated in the scene feature data comprise visual features at a first intersection point of a light ray, cast through the pixel, with an object and/or surface in the scene.
11. A computer-implemented method according to any preceding claim, wherein the scene feature data for the pixel is derived using a geometry buffer, G-buffer, obtained using a rasterization process.
12. A computer-implemented method according to any preceding claim, wherein the ANN is trained using a quality score indicative of a visual quality of images rendered using budget allocation parameters determined by the ANN.
13. A computer-implemented method according to any preceding claim, wherein the ANN is trained based on a comparison between budget allocation parameters determined by the ANN for rendering pixels of a training image and predetermined budget allocation parameters for rendering the pixels of the training image.
14. A computing device comprising: one or more processors; and memory, wherein the computing device is arranged to perform, using the one or more processors, a method according to any preceding claim.
15. A computer program product arranged, when executed on a computing device comprising one or more processors and memory, to cause the computing device to perform, using the one or more processors, a method according to any of claims 1 to 13.

Description

Technical Field The present disclosure concerns computer-implemented methods for rendering images of three-dimensional scenes. In particular, but not exclusively, the disclosure concerns computer-implemented methods, computing devices and computer program products for rendering images of three-dimensional scenes using path tracing. Background Rendering images or videos is a key component in many applications. For example, online gaming or virtual reality, VR, applications, which are increasingly popular forms of entertainment and social activity, involve the rendering of images of videos for display to a user. Rendering is a process of generating an image of a scene, which may include one or more three-dimensional models, e.g. representing objects in the scene. Light transport simulation using path tracing may be used in rendering to generate photorealistic images by simulating the way light interacts with objects. The paths of many rays of light are traced as they travel through a scene, reflecting, refracting, and scattering until they eventually hit a light source or fade away. This technique is widely used in computer graphics, particularly in movie production, architectural visualization, and video game development, due to its ability to produce high-quality images that closely resemble real-world lighting. Rasterization, a more traditional and computationally efficient rendering technique compared to path tracing, does not address complex light interactions such as indirect lighting, caustics, soft shadows, and colour bleeding, whereas path tracing produces all of these effects naturally due to a realistic light transport simulation. However, path tracing may be computationally expensive and time-consuming. The primary challenge arises from the need to trace a large number of paths to accurately capture the complex interactions of light in a scene. Each pixel in the image may require hundreds or thousands of samples (i.e. traced light paths) to reduce noise and achieve a visually appealing result. As an example, rendering an animated high-definition (HD) scene at 60 frames per second (fps) using path tracing with 100 samples per pixel (spp) requires 1280 × 720 × 100 × 60 = 5,529,600,000 traced paths per second. This intensive computational demand can make path tracing impractical for real-time or low latency applications. The present disclosure seeks to solve or mitigate some or all of these above-mentioned problems. Alternatively and/or additionally, aspects of the present disclosure seek to provide improved methods for rendering images of three-dimensional scenes. Summary In accordance with a first aspect of the present disclosure there is provided a computer-implemented method for rendering an image of a three-dimensional scene using path tracing, the method comprising, for a pixel of the image to be rendered using path tracing: obtaining scene feature data for the pixel, the scene feature data indicating visual features of a location of the three-dimensional scene for depiction by the pixel in the image;processing, at an artificial neural network, ANN, the scene feature data to determine a budget allocation parameter for rendering the pixel using path tracing, the budget allocation parameter indicative of an amount of computing resources to be used for rendering the pixel using path tracing, wherein the ANN is trained to determine, from the scene feature data, the budget allocation parameter based on the visual features indicated by the scene feature data; andoutputting, from the ANN, the determined budget allocation parameter to control a rendering of the pixel using path tracing. As such, computing resources for path tracing are distributed across pixels in a more intelligent manner, thereby increasing computational efficiency. For example, certain areas of an image may require more path tracing samples (or longer light paths) to accurately depict complex lighting effects such as caustics, shadows, and reflections, while other areas may need fewer path tracing samples (or shorter light paths) due to simpler lighting conditions. The complexity of a particular pixel of the image may be represented by the scene feature data for the pixel. Known path tracing methods typically allocate a uniform number of path tracing samples (or a uniform maximum light path length) across all pixels, leading to inefficiencies where computational resources are wasted on areas that do not need as much detail, while other areas remain under-sampled and noisy. The presently-disclosed methods allow a trade-off between computational resources and image quality to be navigated in the following ways. First, for a given overall computational budget (e.g. the total number of path tracing samples available) the presently-disclosed methods distribute the samples per pixel optimally such that image quality is maximized. Second, for a given image quality, the presently-disclosed methods can optimize the allocation of samples to p