US-12618947-B2 - Anti flicker filter for dToF sensor

US12618947B2US 12618947 B2US12618947 B2US 12618947B2US-12618947-B2

Abstract

Method, having detecting from a first histogram signal delivered by a sensor device, successive sets of targets at respective successive instants, determining for a current set of current detected targets, a current histogram output, the current histogram output having for each current detected target of the current set, a current group of parameters stored in a memory including a confidence indicator, performing a matching operation between the current set of detected targets and previous sets of detected targets stored in the memory, and performing a filtering operation of at least one parameter of the current group of parameters of at least some of the current detected targets of the current set, on the basis of the result of the matching operation, the filtering operation being weighted on the basis of at least the confidence indicators of current and previous sets of detected targets.

Inventors

Olivier Pothier
Victor MACELA
Thierry Lebihen
Arnaud Bourge

Assignees

STMICROELECTRONICS FRANCE

Dates

Publication Date: 20260505
Application Date: 20220321
Priority Date: 20210325

Claims (20)

1 . A method, comprising: identifying successive sets of targets at respective successive instants from a first histogram signal delivered by a sensor device, the identifying of the successive set of targets comprising identifying pulses in the first histogram signal that exceed a detection threshold, wherein the sensor device comprises a time-of-flight (ToF) sensor, the first histogram signal representing photon event detection counts for a number of time bins; determining a histogram output for a current set of the successive sets of targets, the histogram output having a group of parameters stored in memory for each current set, each group of parameters having a confidence indicator, wherein a group of parameters for the current set includes a rate parameter, an ambient rate parameter, and a distance parameter; performing a matching operation between the current set of the successive sets of targets and previous sets of the successive sets of targets stored in the memory; performing a filtering operation of at least one parameter of the group of parameters of the current set based on results of the matching operation, the filtering operation weighted based on at least confidence indicators of the current set of targets and the previous sets of the successive sets of targets; and generating a stabilized target detection output with reduced temporal flicker effects from filtered parameters.
2 . The method of claim 1 , wherein the confidence indicator of a current set is based on a target z-score in the first histogram signal.
3 . The method of claim 1 , wherein the matching operation comprises using an affinity criterion having a condition based on the rate parameter, the distance parameter, or the confidence indicator.
4 . The method of claim 3 , wherein the affinity criterion includes an affinity-based distance matrix.
5 . The method of claim 1 , wherein the current set of the successive targets matches a previous set of the successive targets stored in the memory at the end of the matching operation.
6 . The method of claim 1 , wherein the current set of the successive targets does not match a previous set of the successive targets stored in the memory at the end of the matching operation.
7 . The method of claim 6 , wherein the current set of the successive targets is added to the memory based on the availability of empty slots in memory.
8 . The method of claim 1 , wherein the method further comprises performing a selective memory replacement based on confidence indicators for storing the current set of the successive targets.
9 . The method of claim 1 , wherein a previous set of the successive targets in the memory is not matched in the current set of successive targets and, based thereon, a virtual target is added to the input of the filtering operation with a new confidence indicator.
10 . The method of claim 9 , wherein the new confidence indicator is very low in response to high confidence in its corresponding previous target.
11 . The method of claim 9 , wherein the new confidence indicator is close to a confidence indicator of the previous set of the successive targets in response to the confidence indicator of the previous set of the successive targets is low.
12 . The method of claim 1 , further comprising a double weighted filtering operation based on target parameters and confidence indicator.
13 . The method of claim 1 , wherein the sensor device includes a direct time-of-flight (ToF) device.
14 . A system, comprising: a sensor device configured to transmit a first histogram signal to identify successive sets of targets at respective successive instants, the identifying of the successive set of targets comprises identifying pulses in the first histogram signal that exceed a detection threshold, wherein the sensor device comprises a time-of-flight (ToF) sensor, the first histogram signal representing photon event detection counts for a number of time bins; a first processor configured to determine a histogram output for a current set of the successive sets of targets, the histogram output having a group of parameters stored in memory for each current set, each group of parameters having a confidence indicator; and a second processor configured to: perform a matching operation between the current set of the successive sets of targets and previous sets of the successive sets of targets stored in the memory, wherein a group of parameters for the current set includes a rate parameter, an ambient rate parameter, and a distance parameter, perform a filtering operation of at least one parameter of the group of parameters of the current set based on results of the matching operation, the filtering operation weighted based on at least confidence indicators of the current set of targets and the previous sets of the successive sets of targets, and generate a stabilized target detection output with reduced temporal flicker effects from filtered parameters.
15 . The system of claim 14 , wherein the confidence indicator of a current set is based on a target z-score in the first histogram signal.
16 . The system of claim 14 , wherein the matching operation comprises using an affinity criterion having a condition based on the rate parameter, the distance parameter, or the confidence indicator.
17 . The system of claim 16 , wherein the affinity criterion includes an affinity-based distance matrix.
18 . The system of claim 14 , wherein the current set of the successive targets matches a previous set of the successive targets stored in the memory at the end of the matching operation.
19 . The system of claim 14 , wherein the current set of the successive targets does not match a previous set of the successive targets stored in the memory at the end of the matching operation.
20 . The system of claim 19 , wherein the current set of the successive targets is added to the memory based on the availability of empty slots in memory.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to French Application No. 2103003, filed on Mar. 25, 2021, which application is hereby incorporated by reference herein in its entirety. TECHNICAL FIELD The present disclosure generally relates to Time-of-Flight (ToF) and, in particular embodiments, to an anti-flicker filter used in ToF devices. BACKGROUND Devices for determining distance (or range) to objects or targets are known. One such method is called “Time of Flight” (ToF), which includes sending a light signal towards the object and measuring the time taken by the signal to travel to the object and travel back to the device. Direct Time of Flight (dToF) devices directly measure the time taken by the signal to travel to the object and travel back to the device. Indirect Time of Flight (iToF) devices calculate the time the signal takes to travel by measuring the phase shift between the signal coming out of the light source and the signal reflected from the object and detected by a light sensor—knowing this phase shift and the speed of light enables the determination of the distance to the object. Single-photon avalanche diodes (SPAD) may be used to detect reflected light. In general, an array of SPADs is provided as a sensor to detect a reflected light pulse. A photon may generate a carrier in the SPAD through the photoelectric effect. The photo-generated carrier may trigger an avalanche current in one or more of the SPADs in the SPAD array. The avalanche current may signal an event, namely that a photon of light has been detected. Time-of-Flight histogram processing aims to detect objects (targets) in the Field-of-View of the device and estimate their distances to the sensor. This processing (Histogram Pipe “HIP”) is embedded into the ToF sensor firmware (FW), and it can be used by a Smartphone to drive its Camera AutoFocus. For example, the HIP uses a statistical hypothesis test to determine the presence of potential targets in the field of view: it is the role of a Pulse Segmenter (“PS”) or pulse detector. The PS finds statistically significant pulses in the Histograms reported by the hardware (HW) of the ToF (for example, dToF) sensor. Due to the statistical nature of this detection process and the Photonic noise, targets can “flicker” (i.e., they can be detected at one instant), then lost at the next one, then come back. The flicker is inherent to the technology. Flicker is a temporal effect when the signal bounced by the target is close to the detectability threshold because (i) the target is far away, (ii) the target reflectance is low, or (iii) the target is only covering a small part of the Field of View (FoV) (of one sensor's zone). This is known as a type 1 flicker. Some other types of flicker may occur, such as “Competition” of several targets with equivalent signals when one only has to be reported (type 2), or close targets can be reported successively either as separated or merged (type 3) Several tracks have been followed to reduce the flicker effects. A first track tries to avoid the issue by avoiding statistical detection when not required. However, it is not possible in all cases because AutoFocus requires target distance identification. A second track tries to face the issue using a hysteresis filter (dynamic adaptation of the PS threshold), which reduces the temporal effect but does not remove it. The “curse of Threshold” remains, and it does not solve flickers of types greater than 1. Further, a latency is introduced. A third track tries to face the issue by using and exploiting spatial correlation. But it is not possible in all cases, for example, for Single Zone Sensor, and there is further a loss in (X, Y) resolution. SUMMARY According to embodiments, a flicker filtering method and system having the following advantages is proposed: (i) tight hardware coupling for target confidence computation (done in the firmware), (ii) the Anti Flicker Filter (AFF) allows to drastically reduce the flicker at the output of the Histogram Pipe, (iii) keeping the latency to a reasonable level, (iv) tunable trade-off between the stability and the latency, (v) smoothly handling multiple targets, (vi) detecting False Long Tail targets, (vii) not reducing the spatial resolution (nor making an assumption on the scene). According to embodiments, it is proposed to use a confidence measure on the detected targets. This confidence measure is advantageously computed within the HIP, in FW, using HW histograming counters by performing a temporal filtering (for taking into account the temporal effect of the flickering) with a “short term” memory (Direct part) and a “long term” memory (Feedback part) using weights of the filter having a dynamic and double influence and giving more importance to more recent past than older (temporal influence) and giving more importance to more confident data (confidence influence), dealing with missing targets for “filling” the holes in the filter input