Search

EP-4150522-B1 - AMBIENT LIGHT SUPPRESSION

EP4150522B1EP 4150522 B1EP4150522 B1EP 4150522B1EP-4150522-B1

Inventors

  • BOURQUIN, Yannyk, Parulian, Julian
  • Cuillery, Emilie
  • Damodaran, Mathivanan
  • PALERO, Jonathan, Alambra

Dates

Publication Date
20260506
Application Date
20210507

Claims (15)

  1. A system (200) for performing ambient light suppression, the system comprising: an image projection unit (210) configured to project an illumination pattern onto at least a portion of a scene, wherein the illumination pattern is a time-varying spatially modulated pattern having a predetermined spatial frequency; an imaging unit (220) having a field of view, the imaging unit (220) being configured to capture a plurality of images of the scene while the time-varying spatially modulated illumination pattern having the predetermined spatial frequency is projected onto the scene; and a processing unit (230) configured to demodulate the plurality of images based on the illumination pattern, characterized in that the processing unit is configured to demodulate the plurality of images with respect to a target section in the plurality of captured images, wherein the target section corresponds to one of: - a portion of the scene on which the illumination pattern is selectively projected while the plurality of images were captured; - a portion of the scene at which the projected illumination pattern is resolvable; and - a portion of the scene with pixel depth values which satisfy a predetermined range, in that the target section is a part of the field of view of the imaging unit, in that the processing unit is configured to generate an ambient light suppressed image of the scene based on results of the demodulation, and in that elements of the scene which are outside the target section are excluded from the ambient light suppressed image.
  2. The system (200) according to claim 1, wherein the image projection unit (210) is configured to only selectively project the illumination pattern onto a selected portion of the field of view of the imaging unit (220), and the target section corresponds to a portion of the scene on which the illumination pattern is selectively projected, and wherein the processing unit (230) is configured to demodulate the plurality of images with respect to the target section such that the generated ambient light suppressed image of the scene only depicts one or more elements included in the selected portion of the field of view of the imaging unit.
  3. The system (200) according to claim 1, wherein imaging unit (220) is configured to capture the plurality of images of the scene at a predetermined focal depth, the predetermined spatial frequency of the illumination pattern and the predetermined focal depth of the imaging unit being selected such that the illumination pattern is only resolvable within a certain distance from the focus of the imaging unit, and wherein the target section corresponds to a portion of the scene at which the projected illumination pattern is resolvable, and further wherein the processing unit (230) is configured to demodulate the plurality of images with respect to the target section such that the generated ambient light supressed image of the scene only depicts one or more elements included in field of view within the certain distance range from the focus of the imaging unit.
  4. The system (200) according to claim 1, wherein the processing unit (230) is configured to analyse the plurality of images to determine 3D depth information of the scene, wherein the 3D depth information comprises depth values for each of the pixels of the plurality of images and the target section is based on the 3D depth information of the scene, wherein the processing unit (230) is configured to generate the ambient light suppressed image of the scene by only outputting the demodulation results with respect to the target section.
  5. The system (200) according to claim 4, wherein the processing unit (230) is configured to determine the target section by applying a phase mask to the plurality of images.
  6. The system (200) according to any one of the preceding claims, wherein the illumination pattern comprises a phase-shifting sinusoidal pattern, and the imaging unit (220) is configured to capture the plurality of images at a predetermined phase difference with respect to the phase of the sinusoidal pattern.
  7. The system (200) according to claim 6 when dependent on claim 4 or claim 5, wherein the illumination pattern further comprises at least one phase ramp of a predetermined step size.
  8. The system (200) according to any one of the preceding claims, wherein the imaging unit (220) is configured to capture, while the illumination pattern is projected onto the scene, three sets of images, wherein a first set of the three sets of images corresponds to a 0° phase shift of the sinusoidal pattern, a second set of the three sets of images corresponds to 120° phase shift of the sinusoidal pattern, and a third set of the three sets of images corresponds to 240° phase shift of the sinusoidal pattern.
  9. The system (200) according to claim 8, wherein the imaging unit (220) comprises a colour camera, and each of the first, second, and third set of images comprises a single image.
  10. The system (200) according to claim 8, wherein each of the first, second and third set of images comprises three images, wherein the first image in each of the three of sets of images corresponds to the red colour channel, the second image in each of the three sets of images corresponds to the green colour channel, and the third image in each of the three sets of images corresponds to the blue colour channel.
  11. The system (200) according to any one of the preceding claims, wherein the processing unit (230) is configured to demodulate the plurality of images to produce a first image corresponding to the AC component and a second image correspond to the DC component, wherein the first image is selected as the ambient light suppressed image of the scene.
  12. A method for controlling a system to perform ambient light suppression, wherein the system comprises an image projection unit, an imaging unit, and a processing unit, the method comprising: projecting (302), by the image projection unit, an illumination pattern onto at least a portion of a scene, wherein the illumination pattern is a time-varying spatially modulated pattern having a predetermined spatial frequency; capturing (304), by the imaging unit, a plurality of images of the scene while the time-varying spatially modulated illumination pattern is projected onto the scene; and demodulating (306), by the processing unit, the plurality of images based on the illumination pattern, characterized in that the demodulating is with respect to a target section in the plurality of captured images, wherein the target section corresponds to one of: - a portion of the scene on which the illumination pattern is selectively projected while the plurality of images were captured; - a portion of the scene at which the projected illumination pattern is resolvable; and - a portion of the scene with pixel depth values which satisfy a predetermined range, in that the target section is a part of the field of view of the imaging unit, in that the method comprises generating (308), by the processing unit, an ambient light suppressed image of the scene based on results of the demodulation, and in that the method comprises excluding elements of the scene which are outside the target section from the ambient light suppressed image.
  13. The method according to claim 12, wherein projecting (302) the illumination pattern onto at least a portion of the scene comprises only selectively projecting the illumination pattern onto a selected portion of the field of view of the imaging unit, and wherein the target section corresponds to a portion of the scene on which the illumination pattern is selectively projected, and wherein demodulating (306) the plurality of images with respect to the target section is performed such that the generated ambient light suppressed image of the scene only depicts one or more elements included in the selected portion of the field of view of the imaging unit.
  14. The method according to claim 12, wherein the capturing (304) the plurality of images of the scene is performed at a predetermined focal depth, wherein the predetermined spatial frequency of the illumination pattern and the predetermined focal depth of the imaging unit are selected such that the illumination pattern is only resolvable within a certain distance from the focus of the imaging unit, and wherein the target section corresponds to a portion of the scene at which the projected illumination pattern is resolvable, and further wherein demodulating (306) the plurality of images with respect to the target section is performed such that the generated ambient light supressed image of the scene only depicts one or more elements included in field of view within the certain distance range from the focus of the imaging unit.
  15. A computer program product comprising a computer readable medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform the method according to any one of claims 12 to 14.

Description

FIELD OF THE INVENTION The present disclosure relates to a system for performing ambient light suppression and a method for controlling thereof. BACKGROUND OF THE INVENTION There has been considerable development and investment in exploratory activities in digital innovation in the field of non-obtrusive measurement and monitoring, specifically on skin sensing for personal care and health applications. Currently known skin measurement systems promise skin quantification and monitoring of features of skin that offer consumers information related to changes that may be too small to detect, too faint to notice, and/or too slow to follow. For these systems to be acceptable by consumers, sensing methods and systems should be sensitive as well as specific. Additionally, robustness of measurement is essential to build consumer trust. One critical issue associated with such imaging-based systems, when placed in an uncontrolled environment (e.g. at home) is the undefined and potentially varying ambient lighting. Modulated imaging techniques, such as spatial frequency domain imaging (SFDI), are techniques that use projections of specific light patterns, phase shifted sinusoid patterns mainly, to generate images that can be used for analysis of skin properties for instance. Three spatially modulated images with the same sinusoidal pattern, but phase-shifted, are sufficient to recreate a demodulated AC image where all the DC components of light is excluded, therefore removing the ambient light. Demodulation requires three images of the object of interest I1, I2, and I3 recorded with projection of the sinusoidal pattern with the same spatial frequency, but with 23π phase difference each (0, 23π,43π). The demodulation of the images can be represented by the formulae (1) and (2) below: MAC=23I1−I22+I2−I32+I1−I321/2 MDC=I1+I2+I3/3 where MAC is the AC component of the image (which can be regarded to correspond to the modulated illumination), and MDC is the DC component of the image (which can be regarded to correspond to the ambient illumination). As an example, Fig. 1 illustrates an ambient light correction operation by way of a number of images. Specifically, Fig. 1 includes an initial image 110, a plurality of modulated images 120A, 120B, and 120C, as well as demodulated images 130 and 140 which shows how ambient light can be corrected for based on a number of modulated images. In the context of this disclosure, the term "modulated image" may refer to an image which depicts a modulated pattern being projected onto a part of a scene, and the term "demodulated image" may refer to an image that underwent demodulation with respect to the depicted modulated pattern. In the initial image 110, the scene is only illuminated by ambient light and it can be seen that the face is unevenly illuminated. Each of a first modulated image 120A, a second modulated image 120B, and a third modulated image 120C is associated with a different phase shift. In this example, the first modulated image 120A is associated with a 0° phase shift, the second modulated image 120B is associated with a 120° phase shift, and the third modulated image 120C is associated with a 240° phase shift. The uneven illumination that is presented in the initial image 110 can be corrected by performing demodulation of the modulated images 120A, 120B, and 120C according to formulae (1) and (2) as presented above, so as to generate an AC component 130 and a DC component. Specifically, the three sinusoidal patterns captured in the three modulated images 120A, 120B, and 120C are demodulated to arrive at the demodulated images 130 and 140, which respectively correspond to the AC component 130 (representing the alternating part of the demodulated signal) and the DC component 140 (representing the constant part of the demodulated signal). In this case, the AC component 130 corresponds to the modulated illumination while the scene is illuminated with both the projected modulated illumination and ambient illumination, while the DC component 140 of the image corresponds to the ambient illumination while the scene is illuminated with both the modulated illumination and ambient illumination. Accordingly, the AC component 130 may be regarded as an "ambient light corrected/suppressed" version that represents the scene depicted in the initial image 110. It is noted that US patent application US 2019/0101383 A1 discloses a technique for determining an object using structured light to overcome ambient light effects. The technique according to US 2019/0101383 A1 utilizes structured light of various spatial frequencies. It is further noted that Bodenschatz et al. in their paper "Diffuse optical microscopy for quantification of depth-dependent epithelial backscattering in the cervix" (Journal of Biomedical Optics (vol. 21, no. 6, 1 June 2016) discuss the use of structured light of different spatial frequencies to make observation at different tissue depths. US 2016/004145 discloses a proj