US-12627764-B2 - Vision sensor, image processing device including the same, and operating method of the vision sensor

Abstract

Provided are a vision sensor, an image processing device including the same, and an operating method of the vision sensor. The vision sensor includes a pixel array including a plurality of pixels, a synchronization control module configured to receive a synchronization signal from an external image sensor and generate an internal trigger signal based on the synchronization signal, an event detection circuit configured to receive the internal trigger signal and detect whether an event has occurred in the plurality of pixels and generate event signals corresponding to pixels in which the event has occurred, and an interface circuit configured to receive the event signals and transmit to an external processor vision sensor data based on the trigger signal and at least one of the event signals, where the vision sensor data includes matching information for timing image frame information generated by the image sensor and the event signals generated by the vision sensor.

Inventors

• Jongseok Seo
• Jaeho BAEK

Assignees

• SAMSUNG ELECTRONICS CO., LTD.

Dates

Publication Date

20260512

Application Date

20230522

Priority Date

20201201

Claims (20)

1. 1 . An image processing device comprising: a complementary metal-oxide-semiconductor image sensor (CIS) configured to generate image data of an object; and a vision sensor configured to receive a synchronization signal from the CIS and generate event signals of the object and a timestamp based on the synchronization signal, wherein the CIS is configured to transmit the image data to a single processor, and wherein the vision sensor is configured to transmit the event signals and timestamp to the single processor aperiodically.
2. 2 . The image processing device of claim 1 , wherein the CIS and the vision sensor are implemented as a single chip.
3. 3 . The image processing device of claim 1 , wherein the CIS, the vision sensor, and the processor are implemented as a single chip.
4. 4 . The image processing device of claim 1 , wherein the vision sensor further comprises a voltage generator configured to generate threshold voltages to be used to detect events, and wherein the voltage generator is configured to change voltage levels of the threshold voltages.
5. 5 . The image processing device of claim 1 , wherein the timestamp includes information of time where the event signals are generated.
6. 6 . The image processing device of claim 1 , wherein the CIS is configured to generate the synchronization signal based on at least one of shutter signal information, read signal information and image frame information.
7. 7 . The image processing device of claim 4 , wherein the voltage generator is configured to change the voltage levels of the threshold voltages to be provided to pixels of a region of interest (ROI).
8. 8 . The image processing device of claim 7 , wherein the voltage generator is configured to differently change the voltage levels of the threshold voltages for each of a plurality of ROI.
9. 9 . The image processing device of claim 1 , wherein the vision sensor further comprises an event detection circuit configured to generate a plurality of event signals between a period of a first image frame information of the CIS and a second image frame information of the CIS.
10. 10 . An image processing device comprising: a complementary metal-oxide-semiconductor image sensor (CIS) configured to generate image data of an object; and a vision sensor including: a first plurality of pixels configured to generate events in response to a change in light, the events being related to a movement of the object, and an event detection circuit configured to generate event signals based on the events and a synchronization signal received from the CIS, wherein the CIS is configured to transmit the image data to a single processor, wherein the event detection circuit is configured to process the events in a frame unit, wherein the vision sensor is configured to transmit the event signals and timestamp to the single processor, wherein the event detection circuit is configured to generate event signals N times between a period of a first image frame information of the CIS and a second image frame information of the CIS, and wherein N is an integer greater than 1.
11. 11 . The image processing device of claim 10 , wherein the vision sensor further comprises a voltage generator configured to generate threshold voltages to be used to detect the events, and wherein the voltage generator is configured to change voltage levels of the threshold voltages.
12. 12 . The image processing device of claim 10 , wherein the CIS and the vision sensor are implemented as a single chip.
13. 13 . The image processing device of claim 10 , wherein the CIS is configured to generate the synchronization signal based on at least one of shutter signal information, read signal information, and image frame information.
14. 14 . The image processing device of claim 10 , wherein the timestamp includes information of time where the event signals are generated.
15. 15 . The image processing device of claim 11 , wherein the voltage generator is configured to differently change the voltage levels of the threshold voltages for each of a plurality of regions of interest (ROI).
16. 16 . An image processing device comprising: a complementary metal-oxide-semiconductor image sensor (CIS) configured to generate image data of an object; and a vision sensor including: a first plurality of pixels configured to generate events in response to a change in light, the events being related to a movement of the object, and an event detection circuit configured to generate event signals based on the events and a synchronization signal received from the CIS, wherein the CIS and the vision sensor are configured to transmit the image data and the event signals to a single processor, wherein the vision sensor is configured to transmit the event signals to the single processor periodically, and wherein the event detection circuit is configured to process the events in a frame unit.
17. 17 . The image processing device of claim 16 , wherein the vision sensor further comprises a voltage generator configured to generate threshold voltages to be used to detect the events, and wherein the voltage generator is configured to change voltage levels of the threshold voltages.
18. 18 . The image processing device of claim 16 , wherein the CIS is configured to generate the synchronization signal based on at least one of shutter signal information, read signal information, and image frame information.
19. 19 . The image processing device of claim 16 , wherein the event detection circuit is configured to generate event signals N times between a period of a first image frame information of the CIS and a second image frame information of the CIS, and wherein N is an integer greater than 1.
20. 20 . The image processing device of claim 16 , wherein the CIS and the vision sensor are implemented as a single chip.

Description

CROSS-REFERENCE TO RELATED APPLICATION(S) This U.S. non-provisional patent application is a continuation of co-pending U.S. patent application Ser. No. 17/525,390, titled VISION SENSOR, IMAGE PROCESSING DEVICE INCLUDING THE SAME, AND OPERATING METHOD OF THE VISION SENSOR and filed on Nov. 12, 2021, which, in turn, claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2020-0165942, filed on Dec. 1, 2020, in the Korean Intellectual Property Office, the disclosures of which are incorporated by reference herein in their entireties. FIELD The present disclosure relates to vision sensors, and more particularly, to a vision sensor that may synchronize with a complementary metal-oxide-semiconductor (CMOS) image sensor (CIS) and other devices by using a timestamp, an image processing device including the vision sensor, and an operating method of the vision sensor. DISCUSSION OF RELATED ART When an event (e.g., a change in intensity of light) occurs, a vision sensor, for example, a dynamic vision sensor, generates information about the event, that is, an event signal, and transmits the event signal to a processor. When event signals of the vision sensor are linked to an image sensor, various functions such as de-blur or super slow may be supported. Vision sensors may synchronize in a slave mode by receiving signals from a host, but an increased efficiency method of synchronizing with a CIS or the like may be used as the utilization of vision sensors increases. SUMMARY The present disclosure provides a vision sensor that uses a timestamp to synchronize internal/external devices of a camera including an image sensor, a gyro sensor, and an infrared light-emitting diode (LED), and further provides an image processing device including the vision sensor, and an operating method of the vision sensor. According to an embodiment of the present disclosure, there is provided a vision sensor including a pixel array comprising a plurality of pixels; a synchronization control module configured to receive a synchronization signal from an external image sensor, and generate an internal trigger signal based on the synchronization signal; an event detection circuit configured to receive the internal trigger signal, detect whether an event has occurred in the plurality of pixels, and generate event signals corresponding to pixels in which the event has occurred; and an interface circuit configured to receive the event signals, transmit to an external processor vision sensor data based on the trigger signal and at least one of the event signals, wherein the vision sensor data comprises matching information for timing image frame information generated by the image sensor and the event signals generated by the vision sensor. According to an embodiment of the present disclosure, there is provided an image processing device including an image sensor configured to generate a synchronization signal based on at least one of shutter signal information, read signal information, and image frame information, the shutter signal information comprising information of an exposure time point at which an exposure of a pixel of a pixel array starts or stops, the read signal information comprising information of an operation time point at which a voltage of the pixel of the pixel array is detected, and the image frame information comprising information of a generation time point at which an image frame is generated; a vision sensor configured to generate an internal trigger signal based on the synchronization signal received from the image sensor, generate a plurality of event signals corresponding to a plurality of pixels in which an event has occurred due to a movement of an object among a plurality of pixels comprised in the pixel array, generate a timestamp based on the internal trigger signal, and output vision sensor data comprising at least one of the event signals and the timestamp; and a processor configured to match image sensor data received from the image sensor with the vision sensor data received from the vision sensor. According to an embodiment of the present disclosure, there is provided an operating method of a vision sensor, the method including: generating an internal trigger signal based on a synchronization signal received from an image sensor; detecting whether an event has occurred in a plurality of pixels, generating event signals corresponding to pixels in which the event has occurred, and generating a timestamp based on the internal trigger signal; and communicating with an external processor and transmitting vision sensor data comprising at least one of the event signals and the timestamp to the external processor, wherein the timestamp comprises matching information for timing image frame information generated by the image sensor and the event signals generated by the vision sensor. BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the present disclosure will be more clearly understood from the following det