US-20260127807-A1 - METHODS FOR RENDERING AN IMAGE OF A THREE-DIMENSIONAL SCENE

US20260127807A1US 20260127807 A1US20260127807 A1US 20260127807A1US-20260127807-A1

Abstract

A method for rendering an image of a three-dimensional scene using path tracing. For a pixel of the image to be rendered using path tracing, a budget allocation parameter for rendering the pixel using path tracing is determined. The budget allocation parameter is indicative of an amount of computing resources to be used for rendering the pixel using path tracing. The budget allocation parameter is determined to optimise the entropy of the image of the three-dimensional scene generated using the rendered pixel. The determined budget allocation parameter is output from the ANN to control a rendering of the pixel using path tracing.

Inventors

Matthias Sebastian Treder
Sebastian Alexander Lutz

Assignees

SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED

Dates

Publication Date: 20260507
Application Date: 20251106
Priority Date: 20241107

Claims (15)

1 . A computer-implemented method for rendering an image of a three-dimensional scene using path tracing, the method comprising, for a pixel of the image to be rendered using path tracing: determining a budget allocation parameter for rendering the pixel using path tracing, the budget allocation parameter indicative of an amount of computing resources to be used for rendering the pixel using path tracing, wherein the budget allocation parameter is determined to optimise the entropy of the image of the three-dimensional scene generated using the rendered pixel; and outputting the determined budget allocation parameter to control a rendering of the pixel using path tracing.
2 . A computer-implemented method according to claim 1 , the method comprising rendering the pixel by performing path tracing using the determined budget allocation parameter.
3 . A computer-implemented method according to claim 2 , the method comprising generating the image of the three-dimensional scene using the rendered pixel.
4 . A computer-implemented method according to claim 1 , wherein the entropy of the image of the three-dimensional scene generated using the rendered pixel is optimised to reduce the data required to encode the image using an image codec.
5 . A computer-implemented method according to claim 1 , wherein the image of the pixel is a frame of video, and the entropy of the image of the three-dimensional scene generated using the rendered pixel is optimised to reduce the data required to encode the video using a video codec.
6 . A computer-implemented method according to claim 1 , wherein the budget allocation parameter is indicative of a number of light paths to be traced for rendering the pixel using path tracing.
7 . A computer-implemented method according to claim 1 , wherein the budget allocation parameter is indicative of a maximum path length of light paths to be traced for rendering the pixel using path tracing.
8 . A computer-implemented method according to claim 1 , the method comprising: obtaining a system resource characteristic of a system configured to render the image using path tracing; and using the system resource characteristic and the determined budget allocation parameter to control the rendering, by the system, of the pixel using path tracing.
9 . A computer-implemented method according to claim 6 , wherein the system resource characteristic is time-varying.
10 . A computer-implemented method according to claim 8 , wherein the system resource characteristic is indicative of a total number of light paths to be traced for rendering the image.
11 . A computer-implemented method according to claim 1 , wherein the budget allocation parameter is determined using an artificial neural network, ANN.
12 . A computer-implemented method according to claim 11 , the method comprising receiving, at the ANN, scene feature data for the pixel, the scene feature data indicating visual features of a location of the three-dimensional scene for depiction by the pixel in the image, and wherein the ANN is trained to determine, from the scene feature data, the budget allocation parameter based on the visual features indicated by the scene feature data.
13 . A computer-implemented method according to claim 11 , wherein the ANN is trained using an entropy score indicative of an entropy of images rendered using budget allocation parameters determined by the ANN.
14 . A system comprising: one or more computer processors; and one or more non-transitory computer-readable media that store instructions which, when executed by the one or more computer processors, cause the one or more computer processors to perform operations for rendering an image of a three-dimensional scene using path tracing, the operations comprising: for a pixel of the image to be rendered using path tracing: determining a budget allocation parameter for rendering the pixel using path tracing, the budget allocation parameter indicative of an amount of computing resources to be used for rendering the pixel using path tracing, wherein the budget allocation parameter is determined to optimise the entropy of the image of the three-dimensional scene generated using the rendered pixel; and outputting the determined budget allocation parameter to control a rendering of the pixel using path tracing.
15 . One or more non-transitory computer-readable media that store instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform operations for rendering an image of a three-dimensional scene using path tracing, the operations comprising: for a pixel of the image to be rendered using path tracing: determining a budget allocation parameter for rendering the pixel using path tracing, the budget allocation parameter indicative of an amount of computing resources to be used for rendering the pixel using path tracing, wherein the budget allocation parameter is determined to optimise the entropy of the image of the three-dimensional scene generated using the rendered pixel; and outputting the determined budget allocation parameter to control a rendering of the pixel using path tracing.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to European Application No. 24211537.6, filed Nov. 7, 2024, the contents of which are incorporated herein by reference. TECHNICAL FIELD The present disclosure concerns computer-implemented methods for rendering images of three-dimensional scenes. In particular, but not exclusively, the disclosure concerns computer-implemented methods, computing devices and computer program products for rendering images of three-dimensional scenes using path tracing. BACKGROUND Rendering images or videos is a key component in many applications. For example, online gaming or virtual reality, VR, applications, which are increasingly popular forms of entertainment and social activity, involve the rendering of images of videos for display to a user. Rendering is a process of generating an image of a scene, which may include one or more three-dimensional models, e.g. representing objects in the scene. Light transport simulation using path tracing may be used in rendering to generate photorealistic images by simulating the way light interacts with objects. The paths of many rays of light are traced as they travel through a scene, reflecting, refracting, and scattering until they eventually hit a light source or fade away. This technique is widely used in computer graphics, particularly in movie production, architectural visualization, and video game development, due to its ability to produce high-quality images that closely resemble real-world lighting. Rasterization, a more traditional and computationally efficient rendering technique compared to path tracing, does not address complex light interactions such as indirect lighting, caustics, soft shadows, and colour bleeding, whereas path tracing produces all of these effects naturally due to a realistic light transport simulation. However, path tracing may be computationally expensive and time-consuming. The primary challenge arises from the need to trace a large number of paths to accurately capture the complex interactions of light in a scene. Each pixel in the image may require hundreds or thousands of samples (i.e. traced light paths) to reduce noise and achieve a visually appealing result. As an example, rendering an animated high-definition (HD) scene at 60 frames per second (fps) using path tracing with 100 samples per pixel (spp) requires 1280×720×100×60=5,529,600,000 traced paths per second. This intensive computational demand can make path tracing impractical for real-time or low latency applications. The present disclosure seeks to solve or mitigate some or all of these above-mentioned problems. Alternatively and/or additionally, aspects of the present disclosure seek to provide improved methods for rendering images of three-dimensional scenes. SUMMARY In accordance with a first aspect of the present disclosure there is provided a computer-implemented method for rendering an image of a three-dimensional scene using path tracing, the method comprising, for a pixel of the image to be rendered using path tracing: determining a budget allocation parameter for rendering the pixel using path tracing, the budget allocation parameter indicative of an amount of computing resources to be used for rendering the pixel using path tracing, wherein the budget allocation parameter is determined to optimise the entropy of the image of the three-dimensional scene generated using the rendered pixel; andoutputting the determined budget allocation parameter to control a rendering of the pixel using path tracing. In this way, where budget allocation parameters are used to allocate computing resources to be used for rendering pixels using path tracing, the budget allocation parameters may be determined to optimise the entropy of the images generated using the rendered pixels, rather than the quality or other property of the image. Optimising the entropy of the images allows the data required when they are encoded to be minimised, or equivalently where there is a limit on the bandwidth of data that is available, it allows the available bandwidth to be used optimally, so can for example allow a higher resolution of images to be used. In embodiments, the method comprises rendering the pixel by performing path tracing using the determined budget allocation parameter. The path tracing may be performed by a light simulation model (also referred to as a ‘path tracer’), for example. Where the determined budget allocation parameter is an absolute value (e.g. taking into account a system resource parameter such as the total computation budget for rendering the entire image), the path tracing may be performed using that value directly. Alternatively, where the budget allocation parameter is a relative value (e.g. a value that is independent of the total computation budget available), an absolute or final value may first be calculated, using the budget allocation parameter and a system resource parameter, and the absolute value may