US-12620145-B2 - Luminance-preserving and temporally stable daltonization

Abstract

It is difficult for people with color vision deficiency (CVD) to distinguish between certain colors, e.g., reds and greens may be indistinguishable, causing a loss of information. Image recoloring, daltonization, techniques aim to improve the experience for people with CVD. Preserving luminance between the original image as seen by a person with normal color vision and someone with a CVD assists in preserving image appearance. Conventional algorithms attempt to daltonize images by exploiting the content of the image itself. While this is a suitable idea for an image in isolation, temporal inconsistencies (e.g., flickering) occur when applied to a stream of images, as a color c could be mapped to a color a in one frame and b in another. In contrast, the luminance-preserving technique operates on pixels and provides a consistent mapping and therefore is temporally stable.

Inventors

• Johan Pontus Ebelin
• Cyril CRASSIN
• Tomas Guy Akenine-Möller

Assignees

• NVIDIA CORPORATION

Dates

Publication Date

20260505

Application Date

20231023

Claims (20)

1. 1 . A computer-implemented method, comprising: obtaining an image encoded in a luminance-based color space; projecting a color of a pixel in the image, according to a transform function, to a region in the luminance-based color space to produce a projected color as a first color for the pixel, wherein the region is within a luminance polygon in the luminance-based color space and the region is constrained by the color vision deficiency; and remapping the first color to produce a second color in a recolored version of the image, wherein the second color is constrained by a color vision deficiency (CVD).
2. 2 . The computer-implemented method of claim 1 , wherein the region separates the luminance polygon into a first sub-plane and a second sub-plane, and first color samples within the first sub-plane are remapped to a first portion of the region and second color samples within the second sub-plane are remapped to a second portion of the region.
3. 3 . The computer-implemented method of claim 2 , wherein a gray color point on a luminance axis separates the first portion of the region and the second portion of the region.
4. 4 . The computer-implemented method of claim 2 , wherein the projected color is remapped to a third color within a CVD gamut and, the third color is remapped to a fourth color of the first color samples, and further comprising interpolating between the third color and the fourth color to produce the second color for the pixel.
5. 5 . The computer-implemented method of claim 4 , wherein the third color and the fourth color are interpolated based on at least one distance between the first color and the region.
6. 6 . A computer-implemented method, comprising: obtaining an image encoded in a luminance-based color space; projecting a color of a pixel in the image, according to a transform function, to a region within a displayable color vision deficiency (CVD) gamut in the luminance-based color space to produce a projected color as a first color for the pixel; and remapping the first color to produce a second color in a recolored version of the image, wherein the second color is constrained by a CVD.
7. 7 . A computer-implemented method, comprising; obtaining an image encoded in a luminance-based color space; projecting a color of a pixel in the image, according to a transform function, to a region in the luminance-based color space to produce a projected color as a first color for the pixel; remapping the first color to produce a second color in a recolored version of the image, wherein the second color is constrained by a color vision deficiency (CVD); and for a pixel in the recolored version of the image, selecting the second color from a subset of color samples within the region that are perceived by a CVD observer as equal to the second color and is closest in value to the first color for the pixel.
8. 8 . The computer-implemented method of claim 1 , further comprising: converting an original image in a first color space to the image encoded in the luminance-based color space; and converting the recolored version of the image from the luminance-based color space to the first color space.
9. 9 . A computer-implemented method, comprising: obtaining an image encoded in a luminance-based color space; projecting a color of a pixel in the image, according to a transform function, to a region in the luminance-based color space to produce a projected color as a first color for the pixel; and redistributing color samples within the region to equalize a distribution of the color samples within the region, wherein the first color is remapped to produce a second color in a recolored version of the image and the second color is constrained by a color vision deficiency (CVD).
10. 10 . The computer-implemented method of claim 1 , further comprising storing the second color as a texel in a texture map, wherein the second color is accessed by the first color.
11. 11 . A computer-implemented method, comprising: obtaining an image encoded in a luminance-based color space; projecting a color of a pixel in the image, according to a transform function, to a region in the luminance-based color space to produce a projected color as a first color for the pixel, wherein the region is a line or a plane; and remapping the first color to produce a second color in a recolored version of the image, wherein the second color is constrained by a color vision deficiency (CVD).
12. 12 . A computer-implemented method, comprising: obtaining an image encoded in a luminance-based color space; and remapping first colors in the image to produce second colors in a recolored version of the image, wherein the second colors are constrained by a color vision deficiency (CVD), luminance values of the second colors and the respective first colors are equal, and the remapping of each one of the first colors in the image to produce the second colors is consistent with the remapping of the first colors in one or more additional images in a sequence to produce a temporally stable recolored sequence of images.
13. 13 . The computer-implemented method of claim 1 , wherein at least one of the steps of obtaining, projecting, or remapping is performed on a server or in a data center to generate the recolored version of the image, and the recolored version of the image is streamed to a user device.
14. 14 . The computer-implemented method of claim 1 , wherein at least one of the steps of obtaining, projecting, or remapping is performed within a cloud computing environment.
15. 15 . The computer-implemented method of claim 1 , wherein at least one of the steps of obtaining, projecting, or remapping is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
16. 16 . The computer-implemented method of claim 1 , wherein at least one of the steps of obtaining, projecting, or remapping is performed on a virtual machine comprising a portion of a graphics processing unit.
17. 17 . A system, comprising: a memory that stores an image encoded in a luminance-based color space; and a processor that is connected to the memory, wherein the processor is configured to: project a color of a pixel in the image, according to a transform function, to a region in the luminance-based color space to produce a projected color as a first color for the pixel, wherein the region is within a luminance polygon in the luminance-based color space and the region is constrained by the color vision deficiency; and remap the first color to produce a second color in a recolored version of the image, wherein the second color is constrained by a color vision deficiency (CVD).
18. 18 . A non-transitory computer-readable media storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: obtaining an image encoded in a luminance-based color space; projecting a color of a pixel in the image, according to a transform function, to a region in the luminance-based color space to produce a projected color as a first color for the pixel, wherein the region is within a luminance polygon in the luminance-based color space and the region is constrained by the color vision deficiency; and remapping the first color to produce a second color in a recolored version of the image, wherein the second color is constrained by a color vision deficiency (CVD).
19. 19 . A system, comprising: a memory that stores an image encoded in a luminance-based color space; and a processor that is connected to the memory, wherein the processor is configured to: remap first colors in the image to produce second colors in a recolored version of the image, wherein the second colors are constrained by a color vision deficiency (CVD), luminance values of the second colors and the respective first colors are equal, and the remapping of each one of the first colors in the image to produce the second colors is consistent with the remapping of the first colors in one or more additional images in a sequence to produce a temporally stable recolored sequence of images.
20. 20 . A non-transitory computer-readable media storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: obtaining an image encoded in a luminance-based color space; and remapping first colors in the image to produce second colors in a recolored version of the image, wherein the second colors are constrained by a color vision deficiency (CVD), luminance values of the second colors and the respective first colors are equal, and the remapping of each one of the first colors in the image to produce the second colors is consistent with the remapping of the first colors in one or more additional images in a sequence to produce a temporally stable recolored sequence of images.

Description

CLAIM OF PRIORITY This application claims the benefit of U.S. Provisional Application No. 63/431,585 titled “Luminance-Preserving Temporally Stable Method for Improving Images for People with a Color Vision Deficiency,” filed Dec. 9, 2022 and U.S. Provisional Application No. 63/520,482 titled “Luminance-Preserving Temporally Stable Method for Improving Images for People with a Color Vision Deficiency,” filed Aug. 18, 2023, the entire contents of which are incorporated herein by reference. BACKGROUND Color vision deficiencies (CVDs), more commonly known as color blindness, are often caused by genetics and affect the cones on the retina. Approximately 4.5% of the world's population (8% of males) has some form of CVD. There are many different types and severities of CVDs. Because it is hard for people with CVD to distinguish between certain colors, there might be a severe loss of information when presenting them with images as, e.g., reds and greens may be indistinguishable. To that end, several image recoloring, daltonization, techniques have been proposed, which aim to improve the experience for people with CVD. Conventional daltonization techniques do not preserve luminance, are not temporally stable, and/or cannot be executed in real time. There is a need for addressing these issues and/or other issues associated with the prior art. SUMMARY Embodiments of the present disclosure relate to luminance-preserving and temporally stable daltonization. Systems and methods are disclosed for a daltonization technique that preserves luminance, is temporally stable, and can be executed in real time. In addition, remapped colors are evenly distributed over the color gamut visible to the person with a CVD, reducing the number of similar colors, thus often improving chrominance contrast. The most the severe case of CVD, save for monochromacy (only black-white vision) and complete lack of vision, is dichromacy, where an entire class of cone photopigment is missing. For the dichromat, this means that the three-dimensional RGB gamut becomes two dimensional. A dichromat may see only about 0.4% of the 16 million colors displayable with a 24-bit monitor, which makes the task of improving images using daltonization a challenging one. In fact, trying to present the dichromat with the same experience as someone with normal color vision is impossible. Color confusion is unavoidable; however, achromatic acuity is known to be significantly higher than chromatic acuity, a phenomenon that is exploited by chroma subsampling in image compression algorithms. Thus, a good starting point for preserving image appearance is preserving luminance between the original image as seen by a person with normal color vision and someone with a CVD. Conventional algorithms attempt to daltonize images by exploiting the content of the image itself. While this could be a suitable idea for an image in isolation, it often gives rise to temporal inconsistencies (e.g., flickering) when applied to a stream of images (e.g., video), as a color c could be mapped to a color a in one frame and b in another. In contrast to conventional systems, as described further herein, a luminance-preserving and temporally stable daltonization algorithm is temporally stable, fast, and luminance-preserving. The luminance-preserving and temporally stable daltonization algorithm recolors and presents images in real time. In an embodiment, the method includes obtaining an image encoded in a luminance-based color space and remapping first colors in the image to produce second colors in a recolored version of the image, wherein the second colors are constrained by a color vision deficiency and luminance values of the second colors and the respective first colors are equal. In an embodiment, a first color of a pixel in the image is projected, according to a transform function, to a region constrained by a color vision deficiency and within a luminance polygon in the luminance-based color space to produce a projected color and the projected color is remapped within the region to produce a second color of the pixel in a recolored version of the image. BRIEF DESCRIPTION OF THE DRAWINGS The present systems and methods for luminance-preserving and temporally stable daltonization are described in detail below with reference to the attached drawing figures, wherein: FIG. 1A illustrates an original image and the same image as perceived according to protanopia CVD and an image of relative luminance differences, in accordance with prior art. FIG. 1B illustrates the original image recolored using a conventional technique to produce recolored image and recolored using the luminance-preserving technique to produce a recolored image and images of relative luminance differences, in accordance with an embodiment. FIG. 1C illustrates vertices of an RGB cube transformed into a luminance-preserving space, in accordance with an embodiment. FIG. 2A illustrates sample points within the equi-luminant plane p