Search

EP-4559187-B1 - MULTIPLE-INTENT COMPOSITE IMAGE ENCODING AND RENDERING

EP4559187B1EP 4559187 B1EP4559187 B1EP 4559187B1EP-4559187-B1

Inventors

  • ATKINS, ROBIN
  • PYTLARZ, JACLYN ANNE
  • ZUENA, Jake William

Dates

Publication Date
20260513
Application Date
20230718

Claims (15)

  1. A method (600) of encoding a composite image (506), the method (600) comprising: obtaining (602) a set of constituent images (502) for the composite image (506), the set of constituent images (502) comprising captured images as captured by an imaging sensor which have been adjusted to implement differing respective rendering intents; applying inverse adjustments to the set of constituent images (502) to undo the previously-applied adjustments to implement the differing respective rendering intents and obtain respective reality-render images corresponding to the captured images as captured by the imaging sensor; determining (604) a common rendering intent to be applied to the set of reality-render images; adjusting (606) one or more of the set of reality-render images according to the common rendering intent, resulting in an adjusted set of constituent images (504); creating (608) the composite image (506) based on the adjusted set of constituent images (504); generating (610) metadata (508) characterizing the common rendering intent; and encoding (612) the composite image (506) and the metadata (508) to create an encoded multiple-intent composite image (510).
  2. The method of claim 1, wherein adjusting (606) one or more of the set of reality-render images according to the common rendering intent includes converting a constituent image among the set of constituent images (502) from a present rendering intent to the common rendering intent.
  3. The method (600) of claim 2, wherein converting the constituent image among the set of constituent images (502) from the present rendering intent to the common rendering intent includes: inverting one or more source adjustments of the constituent image; and applying one or more common space adjustments to the constituent image.
  4. The method (600) of claim 3, wherein applying the one or more common space adjustments to the constituent image includes converting sensor values to color values.
  5. The method (600) of claim 3 or claim 4, wherein applying the one or more common space adjustments to the constituent image includes estimating a capture environment surround luminance and white point and applying a white point correction based on the estimated capture environment surround luminance and white point.
  6. The method (600) of any one of claims 3 to 5, wherein applying the one or more common space adjustments to the constituent image includes estimating a capture environment surround luminance and applying an optical-optical transfer function - OOTF - to prepare the image for rendering on a reference display device based in part on the estimated capture environment surround luminance.
  7. The method (600) of any one of claims 3 to 6, wherein applying the one or more common space adjustments to the constituent image includes applying one or more of: a saturation enhancement; a contrast enhancement; individual color saturation adjustments; a slope-offset-power-Tmid enhancement; and tone curve trims.
  8. The method (600) of any one of claims 1 to 7, wherein adjusting one or more of the set of constituent images (502) according to the common rendering intent includes converting a constituent image among the set of constituent images (502) from a preprocessed state to the common rendering intent.
  9. The method (600) of any one of claims 1 to 8, wherein adjusting one or more of the set of constituent images (502) according to the common rendering intent includes: converting a first constituent image among the set of constituent images (502) from a first present rendering intent to the common rendering intent; and converting a second constituent image among the set of constituent images (502) from a second present rendering intent to the common rendering intent.
  10. The method (600) of any one of claims 1 to 9, wherein determining (604) the common rendering intent includes selecting, as the common rendering intent, a present rendering intent of a constituent image among the set of constituent images (502).
  11. The method (600) of claim 10 further comprising selecting the present rendering intent of the constituent image as the common rendering intent based on an importance of the constituent image relative to other constituent images of the set of constituent images (502), wherein the importance of the constituent image is determined based on one or more of: a relative size of the constituent image in a rendered space of the composite image (506); a resolution of the constituent image; a centrality of a location of the constituent image in the rendered space of the composite image (506); and a point of viewer focus in the rendered space of the composite image.
  12. The method (600) of any one of claims 1 to 9, wherein determining (604) the common rendering intent includes identifying a preferred rendering intent as the common rendering intent.
  13. The method (600) of claim 12 further comprising identifying the preferred rendering intent based on one or both of: input received via a user interface; and information in a configuration file.
  14. An apparatus (501A, 800) configured to perform the method (600) of any one of claims 1 to 13.
  15. A computer-readable medium having stored therein instructions that, when executed by one or more processors, cause the one or more processors to perform the method (600) of any one of claims 1 to 13.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority from U.S. Provisional Application No. 63/368,766 filed 18 July 2022, and European Application No. 22213582.4 filed on 14 December 2022. BACKGROUND This application relates generally to systems and methods of image encoding and decoding. US 2016/0005201 A1 discloses systems and methods for overlaying a second image/video data onto a first image/video data. The first image/video data may be intended to be rendered on a display with HDR, EDR, VDR or UHD capabilities. The second image/video data may comprise graphics, closed captioning, text, advertisement or any data that may be desired to be overlaid and/or composited onto the first image/video data. The second image/video data may be appearance mapped according to the image statistics and/or characteristics of the first image/video data. In addition, such appearance mapping may be made according to the characteristics of the display that the composite data is to be rendered. Such appearance mapping is desired to render a composite data that is visually pleasing to a viewer, rendered upon a desired display. Tim Borer et al.: "Perceptual Uniformity for High-Dynamic-Range Television Systems", SMPTE Motion Imaging Journal, vol. 125, no. 8, 1 October 2016, pages 75-84, discloses an HDR television system design based on psychovisual aspects of human vision. The authors review the nonlinearities in conventional SDR television systems and explain how these accommodate psychovisual effects. They discuss brightness perception thresholds according to Weber's and De Vries-Rose's laws in the context of minimizing banding artifacts from quantization, which provides a psychovisual definition for video dynamic range. The paper addresses the effect of surround brightness on perception of emissive images in dim or dark environments, which is accommodated through an overall system nonlinearity called the rendering intent, typically implemented as a gamma function. Based on these psychovisual considerations, the authors propose an HDR television system supporting high quality HDR images with dynamic range substantially beyond human visual system limits in a single adaptation state, defined by a single OETF with variable display EOTF to accommodate eye adaptation, while maintaining compatibility with existing SDR displays and infrastructure without requiring metadata. SUMMARY The invention is defined by the independent claims. The dependent claims concern optional features of some embodiments. At least some aspects of the present disclosure may be implemented via methods. Some methods may involve obtaining a set of constituent images for a composite image, determining a common rendering intent to be applied to the set of constituent images, adjusting one or more of the set of constituent images according to the common rendering intent, resulting in an adjusted set of constituent images, creating the composite image based on the adjusted set of constituent images, generating metadata characterizing the common rendering intent, and encoding the composite image and the metadata to create an encoded multiple-intent composite image. In some examples, adjusting one or more of the set of constituent images according to the common rendering intent can include converting a constituent image among the set of constituent images from a present rendering intent to the common rendering intent. In some examples, converting the constituent image among the set of constituent images from the present rendering intent to the common rendering intent can includes inverting one or more source adjustments of the constituent image and applying one or more common space adjustments to the constituent image. In some examples, applying the one or more common space adjustments to the constituent image can include converting sensor values to color values. In some examples, applying the one or more common space adjustments to the constituent image can include estimating a capture environment surround luminance and white point and applying a white point correction based on the estimated capture environment surround luminance and white point. In some examples, applying the one or more common space adjustments to the constituent image can include estimating a capture environment surround luminance and applying an optical-optical transfer function (OOTF) to prepare the image for rendering on a reference display device based in part on the estimated capture environment surround luminance. In some examples, applying the one or more common space adjustments to the constituent image can include applying one or more of a saturation enhancement, a contrast enhancement, individual color saturation adjustments, a slope-offset-power-Tmid enhancement, and tone curve trims. In some examples, adjusting one or more of the set of constituent images according to the common rendering intent can include converting a constituent image among the set of constitu