CN-122003697-A - Image-specific global tone curve and residual gain map generation based on multiple images

Abstract

Various techniques are disclosed for generating an image-specific global tone curve and residual gain map based on a plurality of images. A gain map is generated based on a comparison of the first image and the second image. A gain map scatter plot is constructed based on the gain map values and channel values of the first image or the second image. An image-specific global tone curve is determined by curve fitting the gain map scatter plot. A residual gain map is generated based on a comparison of the gain map and a reconstructed gain map derived from the image-specific global tone curve. An approximation of the second image can be constructed using the image-specific global tone curve and the first image. A copy of the second image can be constructed using the image-specific global tone curve, the residual gain map, and the first image.

Inventors

• C.E.Huo
• J. K. Roland
• N.P. Bonier

Assignees

• 苹果公司

Dates

Publication Date

20260508

Application Date

20240930

Priority Date

20240927

Claims (20)

1. 1. A method for generating an image-specific global tone curve, the method comprising, at a computing device: Obtaining a gain map based on a comparison between a unit region of a first image and a corresponding unit region of a second image, the gain map comprising a set of gain values for the unit regions of the first and second images; generating a scatter plot comprising a set of points, each point comprising a gain value from the gain map and a brightness value for a corresponding cell region of the first image; determining the image-specific global tone curve based on curve fitting the scatter plot using a cost function, and The image-specific global tone curve is stored with the first image.
2. 2. The method of claim 1, the method further comprising: generating a reconstructed gain map from the image-specific global tone curve, and A residual gain map is determined based on a comparison between the reconstructed gain mapped unit region and a corresponding unit region of the gain map.
3. 3. The method of claim 2, wherein the second image is reproducible using a combination of the first image, the image-specific global tone curve, and the residual gain map.
4. 4. The method of claim 2, wherein the residual gain map comprises a difference between gain values of the reconstructed gain mapped unit region and gain values of corresponding unit regions of the gain map.
5. 5. The method according to claim 1, wherein: The first image comprises a Standard Dynamic Range (SDR) image, and The second image comprises a High Dynamic Range (HDR) image.
6. 6. The method of claim 1, the method further comprising: A variant second image is generated based on the first image and the image-specific global tone curve.
7. 7. The method of claim 6, wherein the variant second image differs from the second image in one or more channel values of a corresponding cell region.
8. 8. The method of claim 1, wherein the image-specific global tone curve comprises a plurality of consecutive segments, each segment corresponding to a cubic spline.
9. 9. The method of claim 1, wherein the curve fitting constrains the image-specific global tone curve to a monotonically increasing function.
10. 10. The method of claim 1, wherein the cost function comprises one of a least squares function, a Structural Similarity Index Metric (SSIM), or a variance expansion factor (VIF).
11. 11. The method of claim 1, wherein the channel values of the first image used to generate the scatter plot comprise luminance values of a cell region of the first image.
12. 12. The method of claim 1, wherein the channel values of the first image used to generate the scatter plot comprise one or more color values of a cell region of the first image.
13. 13. The method of claim 1, wherein each cell region corresponds to a single pixel.
14. 14. The method of claim 1, wherein each cell region corresponds to a plurality of consecutive pixels.
15. 15. A non-transitory computer-readable storage medium configured to store instructions that, when executed by at least one processor included in a computing device, cause the computing device to generate an image-specific global tone curve by implementing steps comprising: Obtaining a gain map based on a comparison between a unit region of a first image and a corresponding unit region of a second image, the gain map comprising a set of gain values for the unit regions of the first and second images; generating a scatter plot comprising a set of points, each point comprising a gain value from the gain map and a brightness value for a corresponding cell region of the first image; determining the image-specific global tone curve based on curve fitting the scatter plot using a cost function, and The image-specific global tone curve is stored with the first image.
16. 16. The non-transitory computer readable storage medium of claim 15, wherein the steps further comprise: generating a reconstructed gain map from the image-specific global tone curve, and A residual gain map is determined based on a comparison between the reconstructed gain mapped unit region and a corresponding unit region of the gain map.
17. 17. The non-transitory computer-readable storage medium of claim 16, wherein the second image is reproducible using a combination of the first image, the image-specific global tone curve, and the residual gain map.
18. 18. The non-transitory computer-readable storage medium of claim 16, wherein the residual gain map comprises a difference between gain values of a unit region of the reconstructed gain map and gain values of a corresponding unit region of the gain map.
19. 19. The non-transitory computer-readable storage medium of claim 15, wherein: The first image comprises a Standard Dynamic Range (SDR) image, and The second image comprises a High Dynamic Range (HDR) image.
20. 20. The non-transitory computer readable storage medium of claim 15, wherein the steps further comprise: A variant second image is generated based on the first image and the image-specific global tone curve.

Description

Image-specific global tone curve and residual gain map generation based on multiple images Technical Field Implementations described herein set forth techniques for generating an image-specific global tone curve and residual gain map based on a plurality of images. In particular, the gain map may be generated based on a comparison of the first image and the second image. A gain map scatter plot may be constructed based on the gain map values and the brightness (channel) values of the first image or the second image. An image-specific global tone curve may be determined by curve fitting a gain map scatter plot, and a residual gain map may be generated based on a comparison of the gain map and a reconstructed gain map derived from the image-specific global tone curve. Background The dynamic range of an image refers to the range of pixel values between the brightest and darkest portions of the image (commonly referred to as "brightness"). Notably, conventional image sensors can capture only a limited range of brightness in a single exposure of a scene, at least with respect to the brightness that the human eye can perceive from the same scene. This limited range is commonly referred to in the digital photography arts as Standard Dynamic Range (SDR). Despite the foregoing image sensor limitations, improvements in photography have enabled capturing of a wider range of light, referred to herein as High Dynamic Range (HDR). This can be accomplished by (1) capturing multiple "bracketing" images, i.e., images having different exposure times, and then (2) combining the bracketing images into a single image that incorporates different aspects of the different exposures. In this regard, a single HDR image has a wider luminance dynamic range than can be captured in each of the individual exposures. This makes HDR images superior to SDR images in several respects. Display devices capable of displaying HDR images (in their true form) are becoming increasingly available as a result of advances in design and manufacturing technology. However, most display devices currently in use (and continuing to manufacture) are only capable of displaying SDR images. Thus, a device with an SDR limited display that receives an HDR image must perform various tasks to convert (i.e., downgrade) the HDR image to an SDR image equivalent. Conversely, devices with HDR-capable displays that receive SDR images may attempt to perform various tasks to convert (i.e., upgrade) the SDR images to HDR image equivalents. Converting an image from an HDR image to an SDR image, or vice versa, using a fixed global tone curve may generate an alternative version of the image that is different from the intention of the creator or initiator of the original image. Accordingly, there is a need for a technique for enabling an image to transition efficiently and accurately between different versions of the image while more accurately maintaining the intent of the image creator. Disclosure of Invention Representative implementations described herein set forth techniques for generating an image-specific global tone curve and residual gain map based on multiple images. A gain map scatter plot may be constructed based on gain map values representing a mapping between pixels (or more generally, cell areas) of a first image and pixels (cell areas) of a second image and brightness (channel) values of the first image or the second image. In some embodiments, the gain map may be a forward map from the first image to the second image. In some embodiments, the gain map may be a reverse map from the second image to the first image. An image-specific global tone curve may be determined by curve fitting a gain map scatter plot, and a residual gain map may be generated based on a comparison of the gain map and a reconstructed gain map derived from the image-specific global tone curve. The exemplary embodiments set forth a method for generating an image-specific global tone curve from a first image and a second image. The method may include (1) obtaining a gain map based on a comparison between pixels (cell areas) of a first image and corresponding pixels (cell areas) of a second image, the gain map including a set of gain values for the pixels (cell areas) of the first image and the second image, (2) generating a scatter plot including a set of points, each point including a gain value and a brightness value, wherein the brightness value may be generated from corresponding channel values for the pixels (cell areas) of the first image, and wherein the gain value represents a mapping between the pixels (cell areas) of the first image and the corresponding pixels (cell areas) of the second image, (3) determining an image-specific global tone curve based on curve fitting the scatter plot using a cost function, and (4) storing the image-specific global tone curve with the first image. In some embodiments, the method further includes (5) generating a reconstructed gain map from the image