Search

CN-121973790-A - Fatigue driving monitoring method, device, equipment and storage medium

CN121973790ACN 121973790 ACN121973790 ACN 121973790ACN-121973790-A

Abstract

The application relates to a fatigue driving monitoring method, a device, equipment and a storage medium. The method comprises the steps of obtaining multi-mode features corresponding to a driver, inputting the multi-mode features into a pre-trained fusion decision network so that the fusion decision network can determine fatigue grades corresponding to the driver and the confidence degrees of the fatigue grades according to the multi-mode features, obtaining vehicle running data corresponding to the vehicle when the confidence degrees of the fatigue grades are smaller than a preset confidence degree threshold value, and controlling the vehicle to execute intervention operation corresponding to the fatigue grades when the vehicle running data corresponding to the vehicle meet preset abnormal running conditions. The method and the device make up the problem that the fatigue state is easy to be misjudged only by image recognition by utilizing the complementarity of the multi-mode features, improve the accuracy of fatigue state recognition, evaluate whether the fatigue level is credible or not by the confidence, introduce vehicle running data for cross verification when the confidence is insufficient, and further improve the accuracy of fatigue state recognition.

Inventors

  • LI XI

Assignees

  • 重庆蓝电汽车科技有限公司

Dates

Publication Date
20260505
Application Date
20251204

Claims (11)

  1. 1. The fatigue driving monitoring method is characterized by comprising the following steps of: Acquiring multi-mode characteristics corresponding to a driver; inputting the multi-modal characteristics into a pre-trained fusion decision network so that the fusion decision network determines the fatigue level corresponding to the driver and the confidence level of the fatigue level according to the multi-modal characteristics; Acquiring vehicle running data corresponding to a vehicle under the condition that the confidence coefficient of the fatigue level is smaller than a preset confidence coefficient threshold value; And under the condition that the vehicle running data corresponding to the vehicle accords with a preset abnormal running condition, controlling the vehicle to execute the intervention operation corresponding to the fatigue level.
  2. 2. The method of claim 1, wherein the step of determining the position of the substrate comprises, The multi-modal features include a head multi-modal feature and/or a physiological multi-modal feature; the method for acquiring the multi-mode characteristics corresponding to the driver comprises the following steps: Collecting normal image data corresponding to the driver, extracting head multi-mode characteristics corresponding to the driver from the preprocessed normal image data, and/or, Collecting physiological data corresponding to the driver, and extracting physiological multi-modal characteristics from the physiological data; the head multimodal feature and/or the physiological multimodal feature are acquired.
  3. 3. The method of claim 2, wherein collecting the normal image data corresponding to the driver comprises: Invoking a camera to acquire image data corresponding to the driver; Identifying the shooting state of the camera according to the image data and/or the state signal corresponding to the camera; Under the condition that the shooting state of the camera is normal shooting, determining the image data as normal image data; And under the condition that the shooting state of the camera is abnormal, discarding the image data, and calling other cameras to continuously acquire the image data corresponding to the driver.
  4. 4. The method of claim 1, wherein controlling the vehicle to perform the intervention operation corresponding to the fatigue level comprises: Controlling the vehicle to execute a sensory cue operation corresponding to the mild fatigue under the condition that the fatigue grade is the mild fatigue; And under the condition that the fatigue grade is moderate fatigue, controlling the vehicle to execute a sensory cue operation corresponding to the moderate fatigue and/or execute a vehicle braking operation corresponding to the moderate fatigue: controlling the vehicle to execute a sensory cue operation corresponding to the severe fatigue and/or execute a vehicle braking operation corresponding to the severe fatigue under the condition that the fatigue grade is severe fatigue; Wherein the prompt intensity of the low-level sensory prompt operation is lower than the prompt intensity of the high-level sensory prompt operation, and the brake intensity of the low-level vehicle brake operation is lower than the brake intensity of the high-level vehicle brake operation.
  5. 5. The method of claim 4, wherein the vehicle braking operation comprises: Determining a standard running speed corresponding to the fatigue grade under the current road condition type, adjusting the current running speed of the vehicle to be less than or equal to the standard running speed in a braking time period corresponding to the fatigue grade under the condition that the current running speed of the vehicle is greater than the standard running speed, and/or, And under the condition that the current driving distance of the vehicle is smaller than the standard driving distance, adjusting the current driving distance of the vehicle to be larger than or equal to the standard driving distance in a braking time period corresponding to the fatigue level.
  6. 6. The method of claim 1, wherein controlling the vehicle to perform the intervention operation corresponding to the fatigue level comprises: Controlling the vehicle to switch the vehicle from a manual driving mode to an automatic driving mode under the condition that the fatigue level is severe fatigue; acquiring front image data corresponding to the vehicle or navigation data corresponding to the vehicle; Identifying a parking area located in front of the vehicle according to the acquired front image data and/or the navigation data; controlling the vehicle to park to the parking area.
  7. 7. The method according to any one of claims 1-6, further comprising: When a preset first redundant switch is detected to be triggered, controlling the vehicle to stop the intervention operation corresponding to the fatigue level being executed, and/or, And when the triggering of the preset second redundant switch is detected, controlling the vehicle to start to execute the intervention operation corresponding to the preset target fatigue level.
  8. 8. The method according to any one of claims 1-6, further comprising: controlling the vehicle to execute intervention operation corresponding to the fatigue level under the condition that the confidence coefficient of the fatigue level is larger than or equal to the confidence coefficient threshold value; and controlling the vehicle to prohibit the execution of the intervention operation corresponding to the fatigue level under the condition that the confidence degree of the fatigue level is smaller than the confidence degree threshold and the vehicle running data corresponding to the vehicle does not accord with the abnormal running condition.
  9. 9. A fatigue driving monitoring device, characterized by comprising: The acquisition module is used for acquiring the multi-mode characteristics corresponding to the driver; the decision module is used for inputting the multi-modal characteristics into a pre-trained fusion decision network so that the fusion decision network can determine the fatigue level corresponding to the driver and the confidence level of the fatigue level according to the multi-modal characteristics; the obtaining module is used for obtaining vehicle running data corresponding to the vehicle under the condition that the confidence coefficient of the fatigue level is smaller than a preset confidence coefficient threshold value; And the control module is used for controlling the vehicle to execute the intervention operation corresponding to the fatigue level under the condition that the vehicle running data corresponding to the vehicle accords with the preset abnormal running condition.
  10. 10. A fatigue driving monitoring device, comprising at least one communication interface, at least one bus connected to the at least one communication interface, at least one processor connected to the at least one bus, at least one memory connected to the at least one bus, wherein the processor is configured to execute a fatigue driving monitoring program stored in the memory to implement the fatigue driving monitoring method of any of claims 1-8.
  11. 11. A computer readable storage medium having stored thereon computer executable instructions that are executed to implement the method of fatigue driving monitoring according to any of claims 1-8.

Description

Fatigue driving monitoring method, device, equipment and storage medium Technical Field The present application relates to the field of vehicle technologies, and in particular, to a method, an apparatus, a device, and a storage medium for monitoring fatigue driving. Background At present, a fatigue driving monitoring system is mainly used for acquiring facial images of a driver based on a camera, carrying out fatigue state identification on the facial images of the driver, and prompting the driver to be in a dangerous driving state currently in an acousto-optic mode when the driver is identified to be in a fatigue state. The existing fatigue driving monitoring system realizes the recognition and prompt of the fatigue state of the driver to a certain extent, but only recognizes whether the driver is in the fatigue state through an image recognition mode, the recognition dimension is single, the adaptability of the image recognition mode to environmental changes (such as illumination and shielding) is poor, the recognition accuracy of the fatigue state is low, and the misjudgment rate is high. Disclosure of Invention The application provides a fatigue driving monitoring method, a device, equipment and a storage medium, which are used for solving the problems that whether a driver is in a fatigue state or not is identified only by an image identification mode, and the identification accuracy of the fatigue state is low. Aiming at the technical problems, the technical scheme of the application is solved by the following embodiments: The embodiment of the application provides a fatigue driving monitoring method, which comprises the steps of obtaining multi-mode features corresponding to a driver, inputting the multi-mode features into a pre-trained fusion decision network so that the fusion decision network can determine fatigue grades corresponding to the driver and confidence degrees of the fatigue grades according to the multi-mode features, obtaining vehicle driving data corresponding to a vehicle under the condition that the confidence degrees of the fatigue grades are smaller than a preset confidence degree threshold value, and controlling the vehicle to execute intervention operation corresponding to the fatigue grades under the condition that the vehicle driving data corresponding to the vehicle accords with preset abnormal driving conditions. The multi-modal feature comprises a head multi-modal feature and/or a physiological multi-modal feature, wherein the multi-modal feature comprises the steps of acquiring normal image data corresponding to a driver and extracting the head multi-modal feature corresponding to the driver from the preprocessed normal image data, and/or acquiring physiological data corresponding to the driver and extracting the physiological multi-modal feature from the physiological data and acquiring the head multi-modal feature and/or the physiological multi-modal feature. The method comprises the steps of collecting normal image data corresponding to a driver, calling a camera to collect the image data corresponding to the driver, identifying the shooting state of the camera according to the image data and/or a state signal corresponding to the camera, determining the image data as the normal image data when the shooting state of the camera is normal, discarding the image data when the shooting state of the camera is abnormal, and calling other cameras to continuously collect the image data corresponding to the driver. The vehicle is controlled to execute the intervention operation corresponding to the fatigue grade, wherein the vehicle is controlled to execute the sensory cue operation corresponding to the mild fatigue under the condition that the fatigue grade is mild fatigue, the vehicle is controlled to execute the sensory cue operation corresponding to the moderate fatigue and/or the vehicle braking operation corresponding to the moderate fatigue under the condition that the fatigue grade is moderate fatigue, the vehicle is controlled to execute the sensory cue operation corresponding to the severe fatigue and/or the vehicle braking operation corresponding to the severe fatigue under the condition that the fatigue grade is severe fatigue, and the prompt intensity of the sensory cue operation of a low grade is lower than the prompt intensity of the sensory cue operation of a high grade, and the braking intensity of the vehicle braking operation of a low grade is lower than the braking intensity of the vehicle braking operation of a high grade. The vehicle braking operation comprises the steps of determining a standard running speed corresponding to the fatigue level under the current road condition type, adjusting the current running speed of the vehicle to be smaller than or equal to the standard running speed in a braking time period corresponding to the fatigue level under the condition that the current running speed of the vehicle is larger than the standard running speed, a