Search

CN-121999472-A - Driver fatigue state image recognition method based on facial video stream analysis

CN121999472ACN 121999472 ACN121999472 ACN 121999472ACN-121999472-A

Abstract

The invention relates to the technical field of intelligent driving assistance and computer vision, in particular to a driver fatigue state image recognition method based on facial video stream analysis. The method comprises the steps of collecting a face video stream of a driver, extracting a face key point sequence of continuous frames and forming manifold feature points, introducing affine invariant Riemann measurement into a symmetrical positive Riemann manifold space, calculating the geodesic distance of the manifold feature points relative to a preset reference awake state point to obtain expression deformation features, constructing an attention entropy field model, calculating the drift speed of the manifold feature points on the manifold surface and converting the drift speed into the entropy increase speed of a system, presetting a potential energy well threshold value, obtaining a fatigue situation assessment result and generating a corresponding driver state instruction. According to the invention, the system can accurately extract the real fatigue characteristics under the condition that the posture of the head of the driver is frequently changed in front of the driver in a non-positive way, and the misjudgment rate caused by the posture change is remarkably reduced.

Inventors

  • CHEN SHUANG

Assignees

  • 贵州理工学院

Dates

Publication Date
20260508
Application Date
20260410

Claims (9)

  1. 1. The driver fatigue state image recognition method based on the facial video stream analysis is characterized by comprising the following steps of: Step one, collecting a driver face video stream, extracting a face key point sequence of a continuous frame, constructing a covariance matrix based on the face key point sequence, and mapping the covariance matrix to a symmetrical positive-definite Riemann manifold space to form manifold feature points; Step two, affine invariant Riemann measurement is introduced into the symmetrical positive-definite Riemann manifold space, and the geodesic distance of the manifold feature point relative to a preset reference awake state point is calculated so as to eliminate Euclidean distance interference caused by rigid movement of the head and obtain expression deformation characteristics; Thirdly, constructing an attention entropy field model, calculating the drift speed of the manifold feature points on the manifold surface, and converting the drift speed into the entropy acceleration rate of the system; And step four, presetting a potential energy well threshold, comparing the entropy increasing rate with the potential energy well threshold to obtain a fatigue situation assessment result, and generating a corresponding driver state instruction according to the assessment result.
  2. 2. The method for recognizing fatigue state images of a driver based on facial video stream analysis according to claim 1, wherein the step one specifically comprises: S11, acquiring the face video stream of a driver through a vehicle-mounted camera, and carrying out face detection and key point positioning on each frame of image to generate a coordinate set containing N key points; s12, calculating the statistical correlation of feature dimensions by using the coordinate set, and constructing the covariance matrix reflecting the local texture and structure distribution of the face; s13, performing positive qualitative verification on the covariance matrix, projecting the covariance matrix into the Riemann manifold space formed by the symmetrical positive definite matrix, and defining the face state of each frame as one manifold feature point on the manifold space.
  3. 3. The method for recognizing fatigue state images of a driver based on facial video stream analysis according to claim 2, wherein the second step specifically comprises: S21, calling the pre-stored reference awake state point, wherein the reference awake state point is the projection of a facial covariance matrix of a driver in the manifold space when the driver is in a standard sitting posture and the attention is focused; S22, applying the affine invariant Riemann metric, and calculating the geodesic distance between the manifold characteristic point of the current frame and the reference awake state point; s23, utilizing the geometric characteristics of the Riemann manifold, processing the head gesture change into parallel movement of the manifold surface, processing the micro-expression change into manifold curvature mutation, quantifying the amplitude of the micro-expression change through the geodesic distance, and outputting the expression deformation characteristics after gesture interference is removed.
  4. 4. A driver fatigue status image recognition method based on facial video stream analysis according to claim 3, wherein the third step specifically comprises: S31, calculating a tangent vector of the manifold feature point at the current moment relative to the previous moment, and acquiring the drift speed of the manifold feature point; S32, mapping the modular length of the tangent vector into a system temperature parameter of the attention entropy field model; And S33, based on the system temperature parameter, solving a local entropy increase at the current moment by combining a thermodynamic equation of the attention entropy field model, and defining the local entropy increase in unit time as the entropy increase rate for representing the dissipation degree of the attention of the driver.
  5. 5. The method for recognizing fatigue state images of a driver based on facial video stream analysis according to claim 4, wherein the fourth step specifically comprises: s41, presetting the potential energy well threshold, wherein the potential energy well threshold comprises a first safety threshold and a second fatigue threshold, and the first safety threshold is smaller than the second fatigue threshold; s42, comparing the entropy increasing rate with the potential energy well threshold value, and executing the following logic judgment: When the entropy acceleration rate is smaller than or equal to the first safety threshold, judging that the driver is in a stable awake state, and generating a monitoring maintaining instruction; When the entropy acceleration rate is larger than the first safety threshold and smaller than or equal to the second fatigue threshold, judging that the driver is in an attention decay transition period, and generating an early warning prompt instruction; and when the entropy acceleration rate is larger than the second fatigue threshold, determining that the driver is in a deep fatigue state, and generating an emergency alarm and intervention instruction.
  6. 6. The method for recognizing fatigue status images of drivers based on facial video streaming analysis according to claim 5, wherein the fourth step further comprises: S43, establishing a circadian rhythm prediction time window, collecting the entropy acceleration rate data in a continuous time period, and constructing an entropy acceleration trend curve; S44, slope analysis is conducted on the entropy increasing trend curve, when the slope is continuously positive and exceeds a preset deterioration trend threshold value, under the condition that the current entropy increasing rate does not exceed the second fatigue threshold value, fatigue risk in a future preset time period is locked, and a predictive intervention signal is generated.
  7. 7. A driver fatigue status image recognition method based on facial video stream analysis according to claim 1, further comprising the step of linearizing using a logarithmic euclidean metric decoder: Projecting the manifold feature points in the symmetrical positive-definite Riemann manifold space to a tangent space by utilizing a logarithmic mapping operator; And classifying or carrying out regression calculation on the projected feature vectors in the cut space by using an Euclidean geometric algorithm to generate linear fatigue confidence score which is used as an auxiliary discrimination basis to be input into the attention entropy field model.
  8. 8. The method for recognizing fatigue state images of a driver based on facial video stream analysis according to claim 2, further comprising an abnormal condition processing step of: monitoring the rank of the covariance matrix in real time; when detecting that the ambient illumination intensity is lower than a preset illumination threshold value or the face shielding area exceeds a preset shielding proportion threshold value, leading the covariance matrix to be rank deficient, automatically stopping Riemann manifold mapping; And switching to a standby characteristic point tracking mode based on Euclidean distance until the covariance matrix is recovered to a full rank state.
  9. 9. The method for recognizing fatigue state images of drivers based on facial video stream analysis according to claim 1, wherein the process of constructing covariance matrix in the first step and calculating geodesic distance in the second step is configured to run on a vehicle-mounted neural network processing unit, and the basis vectors of manifold cut space are stored by using a special video memory, and the operation is accelerated by a matrix eigenvalue decomposition algorithm.

Description

Driver fatigue state image recognition method based on facial video stream analysis Technical Field The invention relates to the technical field of intelligent driving assistance and computer vision, in particular to a driver fatigue state image recognition method based on facial video stream analysis. Background The working flow is generally to collect the video stream of the driver's face, position the key point of the face in each frame of picture, and calculate the linear distance or angle between characteristic points under the Euclidean geometric frame, thus quantify eye closure degree or mouth opening and closing degree, and then judge whether the driver is in fatigue state by comparing the static numerical threshold; However, in the related technology, along with the complex and changeable actual driving conditions, the detection mode based on the Euclidean distance has failure risks caused by head gesture deflection and illumination rapid change, specifically, the large-amplitude rigid movement of the head of a driver can directly change the projection distribution of key points on a two-dimensional plane, so that an algorithm cannot decouple pure expression deformation and head gesture change, thereby causing false detection or missed detection; Accordingly, there is a need for a solution to the problems of the prior art. The above information disclosed in the above background section is only for enhancement of understanding of the background of the disclosure and therefore it may include information that does not form the prior art that is already known to those of ordinary skill in the art. Disclosure of Invention In order to solve the technical problems, the invention discloses a driver fatigue state image recognition method based on facial video stream analysis, and specifically, the technical scheme of the invention is as follows: Step one, collecting a driver face video stream, extracting a face key point sequence of a continuous frame, constructing a covariance matrix based on the face key point sequence, and mapping the covariance matrix to a symmetrical positive-definite Riemann manifold space to form manifold feature points; Step two, affine invariant Riemann measurement is introduced into the symmetrical positive-definite Riemann manifold space, and the geodesic distance of the manifold feature point relative to a preset reference awake state point is calculated so as to eliminate Euclidean distance interference caused by rigid movement of the head and obtain expression deformation characteristics; Thirdly, constructing an attention entropy field model, calculating the drift speed of the manifold feature points on the manifold surface, and converting the drift speed into the entropy acceleration rate of the system; And step four, presetting a potential energy well threshold, comparing the entropy increasing rate with the potential energy well threshold to obtain a fatigue situation assessment result, and generating a corresponding driver state instruction according to the assessment result. Preferably, the first step specifically includes: S11, acquiring the face video stream of a driver through a vehicle-mounted camera, and carrying out face detection and key point positioning on each frame of image to generate a coordinate set containing N key points; s12, calculating the statistical correlation of feature dimensions by using the coordinate set, and constructing the covariance matrix reflecting the local texture and structure distribution of the face; s13, performing positive qualitative verification on the covariance matrix, projecting the covariance matrix into the Riemann manifold space formed by the symmetrical positive definite matrix, and defining the face state of each frame as one manifold feature point on the manifold space. Preferably, the second step specifically includes: S21, calling the pre-stored reference awake state point, wherein the reference awake state point is the projection of a facial covariance matrix of a driver in the manifold space when the driver is in a standard sitting posture and the attention is focused; S22, applying the affine invariant Riemann metric, and calculating the geodesic distance between the manifold characteristic point of the current frame and the reference awake state point; s23, utilizing the geometric characteristics of the Riemann manifold, processing the head gesture change into parallel movement of the manifold surface, processing the micro-expression change into manifold curvature mutation, quantifying the amplitude of the micro-expression change through the geodesic distance, and outputting the expression deformation characteristics after gesture interference is removed. Preferably, the third step specifically includes: S31, calculating a tangent vector of the manifold feature point at the current moment relative to the previous moment, and acquiring the drift speed of the manifold feature point; S32, mapping the modular length of