DE-102024133099-A1 - Method for determining the activity of an eye

Abstract

The invention relates to a method for determining the activity of an eye, comprising at least the following steps performed in the aforementioned order on a computing device: a. based on a video recording, recognition of facial features and definition of a plurality of reference points on each recognized facial feature; b. based on the recognized facial features, locating the pupil and recording the positional progression of the pupil and the reference points; c. using a Kalman filter, based on the recorded position profile, to create a smoothed position profile of the pupil and the reference points; d. using the OPTICS algorithm, sorting points of the smoothed position profile from the reference points; e. based on the sorted points, determining the affinity factors for an affine mapping for translational motion, rotational motion and scaling values; f. Based on the smoothed position profile of the pupil, using the affinity factors to calculate a position profile of the pupil. The method proposed here allows for the determination of good quality of the pupil's positional progression using simple means.

Inventors

• Jonas Brandstetter

Assignees

• DR. ING. H.C. F. PORSCHE AKTIENGESELLSCHAFT

Dates

Publication Date

20260513

Application Date

20241112

Claims (5)

1. Method for determining the activity of an eye (1), comprising at least the following steps performed in the order stated on a computing device (2): a. based on a video recording (3), recognition of facial features (4) and definition of a plurality of reference points (5) at each recognized facial feature (4); b. based on the recognized facial features (4), location of the pupil (6) and recording of the positional profile (7) of each pupil (6) and of the reference points (5); c. using a Kalman filter (10), based on the recorded positional profile (7), creation of a smoothed positional profile (8) of each pupil (6) and of the reference points (5); d. using the OPTICS algorithm (11), sorting of points (12) of the smoothed positional profile (8) of the reference points (5); e. Based on the sorted points (12), determine the affinity factors (13) for an affine mapping for translational motion, rotational motion, and scaling values; f. Based on the smoothed position profile (8) of the pupil (6), calculate a position profile (9) of the pupil (6) using the affinity factors (13).
2. Procedure according to Claim 1 , wherein a calculation using the OPTICS algorithm (11) is performed for each frame (14) of a video recording (3), taking into account a plurality of frames (14) of a first period (15) before and a second period (16) after a respective time (17) in this calculation.
3. Procedure according to Claim 1 or Claim 2 , wherein the only facial features (4) used are the eyelids (18), preferably of a single eye (1).
4. Procedure according to Claim 3 , wherein in step b. the position of the pupil (6) of an eye (1) is estimated using the eyelids (18).
5. Method according to one of the preceding claims, wherein only change values (19) are recorded, preferably exclusively to distinguish between a state of fixation (20) and a saccade (21) of the recorded eye (1).

Description

The invention relates to a method for determining the activity of an eye. Observing and recording the activity of a driver's eye (so-called eye-tracking) is of great importance for driver assistance systems, such as Advanced Driver Assistance Systems (ADAS) at Level 2 (driver in charge) or Level 3 (autonomous driving with permitted driver handover). Current main applications in motor vehicles primarily include attention detection and driver monitoring, drowsiness detection, distraction detection, personalization and comfort functions, gaze-controlled controls, automatic adjustment of settings, and other safety and assistance systems. Vehicles are typically equipped with special infrared cameras that record the driver's face inside the passenger compartment. Often, the driver's face is additionally illuminated by a special LED (Light Emitting Diode). This light source is outside the visible spectrum and is therefore not perceived by the driver. The downstream software components include image processing algorithms, eye detection algorithms, tracking algorithms, and behavioral analysis, often in combination with artificial intelligence. These systems must meet the highest demands for real-time capability and integration into the vehicle architecture. Implementing eye-tracking in a vehicle is complex due to the additional software and hardware required. Existing camera images of the driver in the visible area, such as those from interior monitoring, are rarely used because they often lack the necessary quality. A major reason for this is that the methods used to detect pupils and gaze direction exhibit significant noise (known as jitter). Jitter arises both from the natural movements of the head and thus the eyes, and from the jitter inherent in facial landmark recognition. This noise is a significant problem in video-based (i.e., simplified, a temporal sequence of images, or frames) facial landmark recognition because the position of facial landmarks can fluctuate even when the head is stationary. This can be caused by even minimal changes in lighting conditions or slight shifts in facial pose, leading the algorithm to produce slightly different results for landmark positions, which vary from frame to frame. Based on this, the present invention aims to overcome, at least partially, the disadvantages known from the prior art. The features of the invention are defined in the independent claims, for which advantageous embodiments are shown in the dependent claims. The features of the claims can be combined in any technically meaningful way, whereby the explanations in the following description and features from the figures, which comprise supplementary embodiments of the invention, can also be used. The invention relates to a method for determining the activity of an eye, comprising at least the following steps performed in the aforementioned order on a computing device: a. based on a video recording, recognition of facial features and definition of a plurality of reference points on each recognized facial feature; b. based on the recognized facial features, locating the pupil and recording the positional progression of the pupil and the reference points; c. using a Kalman filter, based on the recorded position profile, to create a smoothed position profile of the pupil and the reference points; d. using the OPTICS algorithm, sorting points of the smoothed position profile from the reference points; e. based on the sorted points, determining the affinity factors for an affine mapping for translational motion, rotational motion and scaling values; f. Based on the smoothed position profile of the pupil, using the affinity factors to calculate a position profile of the pupil. In the preceding and following descriptions, ordinal numbers used serve solely for unambiguous differentiation, unless explicitly stated otherwise, and do not indicate any order or ranking of the components referred to. An ordinal number is a number that is 1000 units long. A nal number greater than one does not necessarily imply that another such component must be present. The method proposed here is particularly suitable for low-quality video recordings, capturing at least one fixation and one saccade (a rapid eye movement, usually between two fixations). These eye movements are involuntary and typically occur in a fraction of a second. Saccades are important for scanning the visual field and efficiently processing visual information. Between saccades, there are fixation phases in which the eye remains still and visual information is processed. Eye tracking analyzes saccades and fixations to understand, for example, how people process visual information, how they react to visual stimuli, and how they perform visual tasks. This information is of great interest in various fields such as psychology, market research, utility studies, and neuroscience. In a motor vehicle, this distinction is particularly important for gaining insights into emotio