CN-122024301-A - Non-contact blink detection method and system based on video stream

CN122024301ACN 122024301 ACN122024301 ACN 122024301ACN-122024301-A

Abstract

The invention discloses a non-contact blink detection method and system based on video streams, comprising the steps of 1, capturing eye video image sequences of a user in real time, calculating average gray values of left and right eye ROIs in each frame of image, forming left and right eye gray value data sequences, 2, maintaining a sliding window inner data sequence with a fixed length L by the left and right eye gray value data sequences, 3, determining an effective blink crest by traversing the sliding window inner data sequence obtained in the step 2 based on threshold parameters of a selected threshold mode, and calculating the normalized depth of the effective blink crest, and 4, carrying out joint judgment on the normalized descending depth of the effective blink crest calculated in the step 3 by adopting a double-threshold mechanism, so as to judge a blink event. The invention solves the problems that the blink detection mode is extremely sensitive to the change of the ambient light, so that misjudgment is easy to cause, and the discrimination capability of the complete form of the blink signal is lacking, so that the robustness is poor.

Inventors

ZHAO YIJIE
ZHANG HAIBO
Ren adan
HU TIANXIN

Assignees

中国航空救生研究所

Dates

Publication Date: 20260512
Application Date: 20251227

Claims (10)

1. A method for non-contact blink detection based on video streaming, comprising; step 1, capturing eye video image sequences of a user in real time, calculating average gray values of left and right eye ROIs in each frame of image, and forming gray value data sequences of left and right eyes; Step 2, the gray value data sequences of the left eye and the right eye maintain a data sequence in a sliding window with a fixed length L; step 3, based on the threshold parameters of the selected threshold mode, determining effective blink wave crests by traversing the data sequences in the sliding window obtained in the step 2, and calculating the normalized depth of the effective blink wave crests; And 4, carrying out joint judgment on the normalized descending depth of the effective blink peak calculated in the step 3 by adopting a double-threshold mechanism so as to judge the blink event.
2. The non-contact blink detection method based on video streaming according to claim 1, wherein the step 1 includes: step 11, capturing an eye video image sequence of a user in real time, and dividing a left eye image and a right eye image for each frame of image; Step 12, dividing a left eye image and a right eye image from each frame of image, respectively detecting and positioning the left eye image and the right eye image, and further extracting eye interested areas ROI of the left eye and the right eye; And step 13, calculating the average gray values of the ROIs of the left eye and the right eye in each frame of image, and buffering the average gray values in time sequence to form a gray value data sequence G_eye [ i ] of the left eye and the right eye, wherein i is the number of the monitored eye.
3. The method of claim 2, wherein the threshold pattern in step 3 comprises a dynamic adaptive threshold pattern and a static preset threshold pattern; The threshold parameters of the dynamic self-adaptive threshold mode are set in real time through the gray value data sequence of the sliding window with the length L maintained in the step 2; The static preset threshold mode uses preset fixed threshold parameters; the threshold parameters of both modes include a minimum peak saliency threshold, a high confidence drop depth threshold, and a low confidence drop depth threshold.
4. The method for non-contact blink detection based on video streaming according to claim 3, wherein the setting of the threshold parameter of the dynamic adaptive threshold mode includes: according to the gray value data sequence of the sliding window maintained in real time in the step 2, calculating the average value mu_open and standard deviation sigma_open of the gray value data sequence in the sliding window, and setting threshold parameters comprises the following steps: 1) A minimum peak saliency threshold of h_min=k1 σ_open; 2) A high confidence degradation depth threshold, t_high=k2 σ_open; 3) Low confidence dip depth threshold t_low=k sigma open; Wherein t_high > h_min > t_low.
5. The method for non-contact blink detection based on video streaming according to claim 4, wherein the searching for the candidate blink peak in step 3 includes: S1, searching a starting point j-1 meeting g_j > g_j-1 in a data sequence in a sliding window, and marking the starting point j-1 as P_start; S2, continuing to traverse backwards, and searching for a local maximum point P_peak of the data sequence in the sliding window, wherein the judgment condition of the P_peak is as follows: g_ { P_peak } > g_ { P_peak-1} and g_ { P_peak } > g_ { P_peak+1}; S3, calculating the significance height of the local maximum point P_peak as follows: ΔH_rise=g_{P_peak}-g_{P_start}; S4, judging whether the found local maximum point P_peak is an effective peak or not, wherein the judging condition is that delta H_rise > H_min, and when the judging condition is met, judging that an effective candidate peak is found, and recording the P_peak as an end point P_rise_end of the rising stage.
6. The method for non-contact blink detection based on video streaming according to claim 5, wherein the searching for the candidate blink peak in step 3 further includes: S5, filtering the invalid wave crest, wherein if the local maximum value point P_peak or P_rise_end is too close to the end of the sequence, the method indicates that the complete falling edge cannot be captured, judges that the found local maximum value point P_peak is the invalid wave crest, and returns a blink-free result; S6, the depth verification and analysis of the descending stage of the data sequence in the sliding window comprise the following steps: S61, starting from the P_rim_end, searching back for the end point P_fall_end of the descending stage, wherein the point meets the following judging conditions that g_ { P_fall_end } < g_ { P_fall_end-1}, and g_ { P_fall_end } < = g_ { P_fall_end+1}; s62, calculating normalized descent depth as follows: In the dynamic self-adaptive threshold mode, calculating a normalized descent depth as: ΔD_fall=(g_{P_rise_end}-g_{P_fall_end})/σ_open; In the static preset threshold mode, Δd_fall directly uses the original difference.
7. The method for non-contact blink detection based on video streaming according to claim 6, wherein the step 4 of performing the joint determination by using a dual threshold mechanism includes: Step 41, judging the high confidence, judging that the blink is effective once when the ΔD_fall > T_high is met, and executing the judgment of step 42 when the ΔD_fall is less than or equal to T_high; Step 42, determining that the blink is valid once when the relationship between Δd_fall > t_low and Δh_rise > h_min is satisfied; Step 43, when the judgment condition of step 42 is not satisfied, it is judged as non-blinking.
8. The video stream-based non-contact blink detection method according to any one of claims 1-7, further comprising: step 5, for the dynamic self-adaptive threshold mode, through online learning and self-adaptive adjustment of threshold parameters, the method comprises the following steps: Step 51, each time a valid blink is confirmed, fine-tuning the estimate of σ_open using the original dip depth and rise height of the blink event as: σ_open=α×σ_open+ (1- α) ×Δd_raw/k, where Δd_raw is the unnormalized decrease depth, α is the learning rate, and k is the scaling factor; step 52, dynamically adjusting the values of the threshold parameters h_min, t_high, and t_low according to the updated σ_open.
9. The non-contact blink detection system based on the video stream is characterized by being used for executing the non-contact blink detection method based on the video stream according to any one of claims 1-8, and comprises an image acquisition and preprocessing module, a data processing and buffering module connected with the image acquisition and preprocessing module, a core calculation and decision module and a system control and output module respectively connected with the data processing and buffering module; the image acquisition and preprocessing module comprises an image sensor, a processing module and a processing module, wherein the image sensor is used for capturing video stream image data comprising the face of a user in real time; The eye detection unit is connected with the image sensor and is used for dividing a left eye image and a right eye image for each frame of image, and detecting and positioning the left eye image and the right eye image so as to position the left eye and the right eye; the ROI extraction and gray level calculation unit is connected with the eye detection unit and is used for extracting eye region of interest (ROI) of the left eye and the right eye according to the positions of the left eye and the right eye, respectively calculating average gray level values of all pixels in the left eye ROI and the right eye ROI and outputting gray level value data of the left eye and the right eye; The data processing and caching module comprises a data buffer area and a sliding window management unit which are connected; the data buffer area is used for buffering the average gray values of the left eye and the right eye of the continuous multi-frame in time sequence to form a gray value data sequence G_eye [ i ] of the left eye and the right eye, wherein i is the number of the monitored eye; the sliding window management unit is used for maintaining a sliding window with a fixed length L for each eye to form a data sequence in the sliding window; the core calculation and decision module comprises a threshold management unit, a waveform morphology analysis unit and a double-threshold joint decision unit which are connected in pairs; The threshold management unit is used for setting, storing and updating threshold parameters in a system and supporting two working modes, wherein the threshold parameters of the dynamic self-adaptive threshold mode are calculated in real time through a gray value data sequence in a sliding window; The waveform morphology analysis unit is respectively connected to the sliding window management unit and the threshold management unit, and is used for traversing the data sequence in the sliding window, identifying the gray level rising starting point P_start and the local maximum point P_peak which meet the conditions, and calculating the peak significance height delta H_rise so as to find an effective candidate blink event; The dual-threshold combined judgment unit is used for receiving the peak significance height delta H_rise and the normalized descending depth delta D_fall from the waveform morphology analysis unit, and adopting a two-level threshold judgment mechanism to carry out final decision so as to judge a blink event; The system control and output module is used for configuring the working mode of the system, initializing the flow, managing coordination and scheduling among the modules, processing the system reset logic and outputting the finally determined blink detection result.
10. The video stream based contactless blink detection system according to claim 9, further comprising a selectable adaptive learning module coupled to the dual threshold decision unit and the threshold management unit, respectively; The adaptive learning module is started in a dynamic adaptive threshold mode of the system, and is used for updating the estimation of the current environment fluctuation degree by utilizing the original descent depth of the blink event after confirming one effective blink every time, namely adjusting sigma_open; The threshold management unit is also used for receiving the updated adjustment sigma_open from the online learning unit and continuously and adaptively adjusting the threshold parameters in the dynamic self-adaptive threshold mode.

Description

Non-contact blink detection method and system based on video stream Technical Field The present invention relates to the field of computer vision and pattern recognition technologies, and in particular, to a non-contact blink detection method and system based on video streaming. Background Blink is an important physiological characteristic and behavior signal, and the automatic detection technology has wide application value. The prior main stream technology mainly comprises: 1. The geometrical feature-based method is to locate the upper and lower eyelids by face keypoint detection techniques and calculate the aspect ratio of the eyes (EyeAspectRatio, EAR). When the EAR value is lower than a preset threshold value, the eye is judged to be closed. The method has higher calculation efficiency, but the accuracy of the method is seriously dependent on the accuracy of key point detection, the stability is rapidly reduced when the head gesture is changed greatly, the illumination is poor or the partial shielding is carried out, and the false alarm rate is high. 2. The deep learning-based method is to judge eye states end to end or directly detect blink events by adopting a Convolutional Neural Network (CNN) and other models. Although the method has high accuracy, a large amount of annotation data is needed for training, the model is complex, the calculation cost is high, and the method is difficult to deploy in an embedded platform with limited calculation resources or a real-time system requiring extremely low delay. 3. A method based on pixel intensity (gray scale) detects blinking by calculating the average gray scale value change of an eye region of interest (ROI). When the eyelid is closed, the pupil and iris are occluded and the average gray level of the area typically varies significantly. However, the existing blink detection method based on gray values is extremely sensitive to changes of ambient light, so that erroneous judgment is easily caused, and the discrimination capability of the complete morphology of blink signals is lacking, so that the robustness is poor. Disclosure of Invention The invention aims to provide a non-contact blink detection method and a non-contact blink detection system based on video streams, which are used for solving the problems that the blink detection mode is extremely sensitive to the change of ambient light, so that misjudgment is easy to occur, and the discrimination capability of the complete form of blink signals is lacking, so that the robustness is poor. The invention provides a non-contact blink detection method based on video stream, which comprises the following steps: step 1, capturing eye video image sequences of a user in real time, calculating average gray values of left and right eye ROIs in each frame of image, and forming gray value data sequences of left and right eyes; Step 2, the gray value data sequences of the left eye and the right eye maintain a data sequence in a sliding window with a fixed length L; step 3, based on the threshold parameters of the selected threshold mode, determining effective blink wave crests by traversing the data sequences in the sliding window obtained in the step 2, and calculating the normalized depth of the effective blink wave crests; And 4, carrying out joint judgment on the normalized descending depth of the effective blink peak calculated in the step 3 by adopting a double-threshold mechanism so as to judge the blink event. Optionally, in the method for detecting blink based on video streaming non-contact as described above, the step 1 includes: step 11, capturing an eye video image sequence of a user in real time, and dividing a left eye image and a right eye image for each frame of image; Step 12, dividing a left eye image and a right eye image from each frame of image, respectively detecting and positioning the left eye image and the right eye image, and further extracting eye interested areas ROI of the left eye and the right eye; And step 13, calculating the average gray values of the ROIs of the left eye and the right eye in each frame of image, and buffering the average gray values in time sequence to form a gray value data sequence G_eye [ i ] of the left eye and the right eye, wherein i is the number of the monitored eye. Optionally, in the non-contact blink detection method based on video stream as described above, the threshold mode in the step 3 includes a dynamic adaptive threshold mode and a static preset threshold mode; The threshold parameters of the dynamic self-adaptive threshold mode are set in real time through the gray value data sequence of the sliding window with the length L maintained in the step 2; The static preset threshold mode uses preset fixed threshold parameters; the threshold parameters of both modes include a minimum peak saliency threshold, a high confidence drop depth threshold, and a low confidence drop depth threshold. Optionally, in the method for detecting a non-contact blink b