Search

US-12621555-B2 - Camera and method for recognizing flashes

US12621555B2US 12621555 B2US12621555 B2US 12621555B2US-12621555-B2

Abstract

A camera is provided for recognizing flashes that has an image sensor having a plurality of pixel elements for generating image information and a control and evaluation unit that is configured to read and evaluate image information from the image sensor as to whether a flash has been detected with the image information. In this respect, the image sensor is an event-based image sensor and the control and evaluation unit is configured to read and evaluate events as image information.

Inventors

  • Romain MÜLLER
  • Dirk STROHMEIER
  • Tijl Schwartzkopff

Assignees

  • SICK AG

Dates

Publication Date
20260505
Application Date
20230926
Priority Date
20220927

Claims (18)

  1. 1 . A camera for recognizing flashes, comprising: an image sensor having a plurality of pixel elements to generate image information; and a control and evaluation unit that is configured to read and evaluate the image information from the image sensor as to whether a flash has been detected by the image information, wherein the image sensor is an event-based image sensor, wherein the control and evaluation unit is configured to read and evaluate events as image information, and wherein the control and evaluation unit is further configured to evaluate the events themselves to determine if the flash was detected without assembling an image therefrom.
  2. 2 . The camera in accordance with claim 1 , wherein the control and evaluation unit is configured to recognize the flash as a minimum number of events within a maximum time.
  3. 3 . The camera in accordance with claim 2 , wherein the minimum number of events is a minimum number of local events.
  4. 4 . The camera in accordance with claim 1 , wherein the image sensor generates image information at an updating frequency of at least one KHz.
  5. 5 . The camera in accordance with claim 1 , wherein a respective pixel element determines when the intensity detected by the pixel element changes and generates an event exactly then.
  6. 6 . The camera in accordance with claim 5 , wherein the event has differential information whether the intensity has decreased or increased.
  7. 7 . The camera in accordance with claim 1 , wherein an event respectively has coordinate information of the associated pixel element, time information, and/or intensity information.
  8. 8 . The camera in accordance with claim 1 , wherein the pixel elements are formed or configured such that an event is only generated at a high contrast difference corresponding to the flash.
  9. 9 . The camera in accordance with claim 1 , wherein the control and evaluation unit is configured only to further evaluate events of increasing intensity.
  10. 10 . The camera in accordance with claim 1 , wherein the control and evaluation unit is configured to discard spatially isolated events, with an event being spatially isolated when too few further events occur in a neighborhood of the event during a maximum time.
  11. 11 . The camera in accordance with claim 10 , wherein the event is spatially isolated when no further events at all occur in a neighborhood of the event during a maximum time.
  12. 12 . The camera in accordance with claim 1 , wherein the control and evaluation unit is configured to compare a number of events occurring during a maximum time with a threshold value and to recognize the flash on an exceeding of the threshold value.
  13. 13 . The camera in accordance with claim 1 , wherein the control and evaluation unit is configured only to associate events within a part zone of a detection zone of the camera with a possible recognized flash.
  14. 14 . The camera in accordance with claim 1 , wherein the control and evaluation unit is configured to determine a location of the flash from the events in which the flash is recognized.
  15. 15 . The camera in accordance with claim 14 , wherein the control and evaluation unit is configured to determine a location of the flash from the events in which the flash is recognized as a mean value, an envelope, or a point cloud of the positions of the pixel elements corresponding to the events.
  16. 16 . A method of recognizing flashes, comprising: generating image information using an image sensor having a plurality of pixel elements; and reading and evaluating the image information from the image sensor as to whether a flash has been detected with the image information, wherein the image sensor is an event-based image sensor, wherein events are read and evaluated as image information, and wherein the events themselves are evaluated to determine if the flash was detected without assembling an image therefrom.
  17. 17 . The camera in accordance with claim 1 , wherein the image sensor generates image information at an updating frequency of at least ten KHz.
  18. 18 . A camera for recognizing flashes, comprising: an image sensor having a plurality of pixel elements to generate image information; and a control and evaluation unit that is configured to read and evaluate the image information from the image sensor as to whether a flash has been detected by the image information, wherein the image sensor is an event-based image sensor, wherein the control and evaluation unit is configured to read and evaluate events as image information, and wherein the control and evaluation unit is configured only to further evaluate events of increasing intensity.

Description

The invention relates to a camera and to a method for recognizing flashes in accordance with the preambles of claim 1 and claim 13 respectively. The recognition of small or larger flashes is a challenge. A flash is here in precise terms not to be understood primarily as a flash of lightning, but rather as a spark discharge such as occurs on a short circuit on a circuit board or generally on a monitored machine. The application case can be a quality inspection or a control monitoring or a timely warning of a risk of fire. Purely optically, a flash means a very brief, very bright, and as a rule locally restricted lighting up between two conductors having an admittedly roughly linear geometry, but one that is unpredictable and rapidly variable in its details. Conventional camera technology is not suitable for the detection of flashes. Due to their very short duration, it is likely that the flashes occur between two shots or frames and are thus overlooked from the start. Working with a correspondingly extended exposure time is not a solution. The flashes may thereby possibly be captured, but the images will be overexposed by other interference effects so that the flashes still remain unrecognizable. U.S. Pat. No. 9,910,082 B2 deals with a device for measuring and detecting electrical discharges. Calibration data on discharges of different strengths are stored in advance here. Camera recordings of discharges are thus evaluated. The document does not look at the problem that the flashes cannot be seen at all in the recordings due to a restricted frame rate. DE 10 2013 106 514 B3 describes a device for safeguarding a machine working in an automated manner in which a camera system delivers camera images of a protected zone that is evaluated for foreign objects. In addition, an analysis of the camera images takes place as to whether a foreign object is a welding spark. Such welding sparks are, however, not an example of a spark discharge or of a flash since it is not a purely electrical phenomenon, but rather a glowing particle, typically within a whole swarm of such particles. Much slower time scales are thus relevant with which conventional cameras deal well. More recently, an innovative camera technique has arisen, the so-called event-based camera. It is also called a neuromorphological camera on the basis of the visual cortex. A conventional camera exposes all of its pixels at a regular frame rate or frame repetition rate and then simultaneously reads them in order thus to acquire a number of stationary two-dimensional images per observed timer period corresponding to the frame rate. There is likewise a matrix of pixels in an event-based camera, but neither a fixed frame rate nor a common reading of pixels. Instead, each pixel individually checks whether it has determined a change in intensity. Image information is only generated and output or read in this case, and indeed only by this pixel. Each pixel is thus a kind of independent motion detector. A detected movement is individually reported as an event. The event-based camera thereby reacts extremely quickly to the dynamics in the scene. Images generated from the events cannot be as intuitively grasped by the human eye because the static image portions are missing An event-based camera is, for example, described in a white paper by Prophesee that can be downloaded from their internet site. Respective pixel circuits for an event-based camera are known from WO 2015/036592 A1, WO 2017/174579 A1, and WO 2018/073379 A1. The paper by Gallego, Guillermo, Henri Rebecq, and Davide Scaramuzza, “A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation”, IEEE Int. Conf. Comput. Vis. Pattern Recog. (CVPR), Vol. 1. 2018 presents methods on how movement, distances, and optical flow can be determined from the data of an event-based camera. US 2013/0335595 A1 discloses an apparatus and a method for event-based image processing, specifically for determining the optical flow that is, as already mentioned, however, a particularly complex and expensive tool. An event-based camera for code reading is used in EP 3 663 963 A1. Even though some reference literature has been named, it is nothing in comparison with the abundance of documents on conventional cameras. Event-based cameras are still little used in practice overall and if they are, then primarily due to their intrinsic movement recognition. The use of an event-based camera has never been considered for flash recognition and nor is movement any suitable description plane for the recognition of flashes. It is therefore the object of the invention to provide a camera having an improved flash recognition. This object is satisfied by a camera and by a method for recognizing flashed in accordance with claim 1 and claim 13 respectively. As already initially mentioned, the term flash stands as a representative of an electrical charge, spark discharge, or a brief a