Search

EP-4352474-B1 - TRANSFORMATION OF COLOR REPRESENTATIONS AND SPECTRAL RECOVERY

EP4352474B1EP 4352474 B1EP4352474 B1EP 4352474B1EP-4352474-B1

Inventors

  • MANGAN, SHMUEL
  • ARAD, Boaz
  • MORAG, Nimrod
  • UNGER, HILIT

Dates

Publication Date
20260506
Application Date
20220607

Claims (9)

  1. A system comprising: at least one hardware processor; and a non-transitory computer-readable storage medium having stored thereon program instructions, the program instructions executable by the at least one hardware processor to: receive a set of spectral datapoints obtained from a plurality of images of natural scenes, wherein the images are captured using a hyperspectral imaging device, perform data sampling to obtain a subset of said set of spectral datapoints that is representative of naturally-occurring spectral samples, project said subset of datapoints over (i) a known spectral response profile of a source color space, and (ii) a known spectral response profile of a target color space, to obtain corresponding datasets of source spectral reference atoms and target spectral reference atoms, receive a digital image in said source color space and, iteratively, for each pixel in said digital image: (a) locate said pixel within said source color space, (b) identify the k nearest atoms to said pixel in said source spectral reference atoms dataset, (c) calculate a transform from said k nearest atoms in said source spectral reference atoms dataset to a corresponding set of k atoms in the target spectral reference atoms dataset, and (d) apply said calculated transform to said pixel, to obtain a target pixel in said target color space, and construct a target image in said target color space from all of said obtained target pixels in said target color space.
  2. The system of claim 1, wherein said transform is a linear transform which maps values between said source color space and said target color space.
  3. The system of claim 1, wherein said transform is calculated as a weighted interpolation, by applying a weighting to each of said of said k nearest atoms in said source spectral reference atoms dataset which is the inverse of the distance of said nearest atom to a point in said source color space representing said pixel.
  4. A computer-implemented method comprising: receiving a set of spectral datapoints obtained from a plurality of images of natural scenes, wherein the images are captured using a hyperspectral imaging device; performing data sampling to obtain a subset of said set of spectral datapoints that is representative of naturally-occurring spectral samples; projecting said subset of datapoints over (i) a known spectral response profile of a source color space, and (ii) a known spectral response profile of a target color space, to obtain corresponding datasets of source spectral reference atoms and target spectral reference atoms; receiving a digital image in said source color space and, iteratively, for each pixel in said digital image: (a) locating said pixel within said source color space, (b) identifying the k nearest atoms to said pixel in said source spectral reference atoms dataset, (c) calculating a transform from said k nearest atoms in said source spectral reference atoms dataset to a corresponding set of k atoms in the target spectral reference atoms dataset, and (d) applying said calculated transform to said pixel, to obtain a target pixel in said target color space; and constructing a target image in said target color space from all of said obtained target pixels in said target color space.
  5. The computer-implemented method of claim 4, wherein said transform is a linear transform which maps values between said source color space and said target color space.
  6. The computer-implemented method of claim 4, wherein said transform is calculated as a weighted interpolation, by applying a weighting to each of said of said k nearest atoms in said source spectral reference atoms dataset which is the inverse of the distance of said nearest atom to a point in said source color space representing said pixel.
  7. A computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions executable by at least one hardware processor to: receive a set of spectral datapoints obtained from a plurality of images of natural scenes, wherein the images are captured using a hyperspectral imaging device; perform data sampling to obtain a subset of said set of spectral datapoints that is representative of naturally-occurring spectral samples; project said subset of datapoints over (i) a known spectral response profile of a source color space, and (ii) a known spectral response profile of a target color space, to obtain corresponding datasets of source spectral reference atoms and target spectral reference atoms; receive a digital image in said source color space and, iteratively, for each pixel in said digital image: (a) locate said pixel within said source color space, (b) identify the k nearest atoms to said pixel in said source spectral reference atoms dataset, (c) calculate a transform from said k nearest atoms in said source spectral reference atoms dataset to a corresponding set of k atoms in the target spectral reference atoms dataset, and (d) apply said calculated transform to said pixel, to obtain a target pixel in said target color space; and construct a target image in said target color space from all of said obtained target pixels in said target color space.
  8. The computer program product of claim 7, wherein said transform is a linear transform which maps values between said source color space and said target color space.
  9. The computer program product of claim 7, wherein said transform is calculated as a weighted interpolation, by applying a weighting to each of said of said k nearest atoms in said source spectral reference atoms dataset which is the inverse of the distance of said nearest atom to a point in said source color space representing said pixel.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority from U.S. Application Ser. No. 63/208,321, filed June 8, 2021. FIELD OF THE INVENTION The invention relates to the field of digital imaging. BACKGROUND Electronic image sensors used for most common imaging purposes rely on measuring electrons produced by the photoelectric effect. An inherent limitation of measuring light in this manner is that chrominance information is lost. Each measured photoelectron indicates only that a photon was absorbed, wherein current electronic photodetectors have no intrinsic process by which to discriminate the wavelength of the absorbed photons, and therefore regards them all as the same. Thus, the basic electronic image sensor is inherently monochromatic. The most common method of color detection in image sensors is the color filter array (CFA), in which a pattern of color filters is overlaid on the image sensor, with a different type of color filter for each individual photodetector. In this way, each photodetector element will only measure light intensity for wavelengths within the passband of the filter. By employing three different types of color filters, an image sensor is able to record intensities at three ranges of wavelengths, which can serve as intensities for three primary colors that together produce a gamut representing many of the colors in the human visible spectrum, for example, the red, green, and blue (RGB) wavelengths, or the cyan, yellow, and magenta (CYM) wavelengths. In contrast, a hyperspectral image comprises values for many more wavelengths. Thus, hyperspectral images provide fine-grained spectral data that can be useful in many applications. However, creating hyperspectral images requires using hyperspectral cameras. Unlike a conventional color camera which typically applies three filter colors to each pixel, a hyperspectral camera may use dozens of filter colors, or a diffraction grating at each image location to scatter light of different colors onto dozens of separate elements of an image sensor. Thus, although hyperspectral images offer many advantages, capturing them requires special hyperspectral cameras, which are typically heavier, bulkier, and expensive. The publication "In Defense of Shallow Learned Spectral Reconstruction from RGB Images" by Aeschbacher J. et al., IEEE International Conference on Computer Vision Workshops, 2017, discloses the calculation or reconstruction of spectral values based on a dictionary of spectral atoms. The foregoing examples of the related art and limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the figures. SUMMARY The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools and methods which are meant to be exemplary and illustrative of isolated aspects of the invention. The scope of the invention is defined by the appended claims. There is provided, in an embodiment, a system comprising at least one hardware processor; and a non-transitory computer-readable storage medium having stored thereon program instructions, the program instructions executable by the at least one hardware processor to: receive a set of spectral datapoints obtained from a plurality of images of natural scenes, wherein the images are captured using a hyperspectral imaging device, perform data sampling to obtain a subset of said set of spectral datapoints that is representative of naturally-occurring spectral samples, and project said subset of datapoints over (i) a known spectral response profile of a source color space, and (ii) a known spectral response profile of a target color space, to obtain corresponding datasets of source spectral reference atoms and target spectral reference atoms. There is also provided, in an embodiment, a computer-implemented method comprising: receiving a set of spectral datapoints obtained from a plurality of images of natural scenes, wherein the images are captured using a hyperspectral imaging device; performing data sampling to obtain a subset of the set of spectral datapoints that is representative of naturally-occurring spectral samples; and projecting said subset of datapoints over (i) a known spectral response profile of a source color space, and (ii) a known spectral response profile of a target color space, to obtain corresponding datasets of source spectral reference atoms and target spectral reference atoms. There is further provided, in an embodiment, a computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions executable by at least one hardware processor to: receive a set of spectral datapoints obtained from a plurality of images of natural scenes, wherein the images are captured using a hyperspectral imagi