CN-117218632-B - Fatigue driving monitoring method, equipment and medium based on cloud intelligent learning

CN117218632BCN 117218632 BCN117218632 BCN 117218632BCN-117218632-B

Abstract

The application discloses a fatigue driving monitoring method, equipment and medium based on cloud intelligent learning, and belongs to the technical field of automobile monitoring. The method comprises the steps of obtaining monitoring video data of a driver and sending the monitoring video data to a cloud, processing the monitoring video data based on an image preprocessing algorithm to extract a plurality of facial features of the driver, processing the plurality of facial features based on a fatigue recognition model to determine fatigue degrees, sending early warning based on the fatigue degrees and sending the fatigue degrees to the cloud, processing a cloud database based on a cloud computing module and carrying out iterative operation on a training recognition model to obtain a training recognition model deviation, outputting the training recognition model with the minimum deviation, processing the training recognition model with the minimum deviation based on a reverse operation algorithm to obtain a weight factor of the training recognition model with the minimum deviation, and updating the fatigue recognition model. The method improves the accuracy, popularity and stability of monitoring the fatigue driving, and does not influence the normal driving of the driver.

Inventors

CHANG JIUPENG
MA YONGJIE
YAN MING
YUAN XIAOLING

Assignees

潍柴新能源商用车有限公司

Dates

Publication Date: 20260512
Application Date: 20230914

Claims (8)

1. The fatigue driving monitoring method based on cloud intelligent learning is characterized by comprising the following steps: the method comprises the steps of obtaining monitoring video data of a driver and sending the monitoring video data to a cloud, wherein the monitoring video data is a video comprising a face and a head of the driver, and the cloud comprises a cloud database and a cloud computing module, and the cloud database is used for storing the monitoring video data; processing the monitoring video data based on a preset image preprocessing algorithm to extract a plurality of facial features of the driver, wherein the plurality of facial features comprise a yawning frequency, a blinking frequency, eyelid closing time, pupil constriction amount, sight line offset out-of-bounds and sight line jump out-of-bounds frequency; Inputting the plurality of facial features as feature parameters into a fatigue recognition model; processing the plurality of facial features based on a hidden layer of the fatigue recognition model to determine a first weight of the plurality of facial features, wherein the first weight is related to a time at which the plurality of facial features appear; matching a preset weight matching table based on the facial features to obtain second weights of different facial features; multiplying the plurality of facial feature first weights by the second weight of the different facial feature to calculate a fatigue value for the driver; comparing the fatigue value of the driver with a preset multi-stage fatigue threshold value to determine the fatigue degree of the driver; Sending out early warning based on the fatigue degree, and sending the fatigue degree to the cloud database; Processing the cloud database based on the cloud computing module, and iteratively computing a training recognition model preset in the cloud computing module to obtain deviation of the training recognition model and output a training recognition model with minimum deviation, wherein the training recognition model and the fatigue recognition model are the same type of neural network model; processing the training recognition model with the minimum deviation based on a preset reverse operation algorithm to obtain a weight factor of each neuron of the training recognition model with the minimum deviation; And processing the weight factor of each neuron based on a preset updating algorithm to update the fatigue identification model.
2. The fatigue driving monitoring method based on cloud intelligent learning according to claim 1, wherein the monitoring video data is processed based on a preset image preprocessing algorithm to extract a plurality of facial features of the driver, and specifically comprises: Processing the monitoring video data based on a preset correction video algorithm, and filtering, enhancing and correcting distortion of the monitoring video data to obtain correction video data; identifying the corrective video data, and segmenting the face of the driver based on the five sense organs of the driver to obtain a plurality of facial features of the driver; Tracking a plurality of facial features of the driver with reference to one of the plurality of facial features, and extracting the facial features of the driver.
3. The fatigue driving monitoring method based on cloud intelligent learning according to claim 1, wherein the cloud database is processed based on the cloud computing module, and a training recognition model preset in the cloud computing module is iteratively operated to obtain a deviation of the training recognition model, and a training recognition model with the minimum deviation is output, and the method specifically comprises: Training the training recognition model by taking the cloud database as a sample to obtain a predicted value of the monitoring video data, wherein the monitoring video data is taken as a characteristic value and the fatigue degree associated with the monitoring video data is taken as a label; comparing the predicted value of the training recognition model with the label to obtain the deviation of the predicted value and the true value; Comparing the deviations of two adjacent training recognition models one by one to output a training recognition model with the minimum deviation; and outputting a training recognition model with the minimum deviation when the deviation meets a preset deviation rule.
4. The fatigue driving monitoring method based on cloud intelligent learning of claim 3, wherein when the deviation meets a preset deviation rule, outputting a training recognition model with the minimum deviation, and specifically comprising: outputting the training recognition model with the minimum deviation when the fatigue recognition model updating interval reaches a preset updating interval threshold value and/or And outputting the training recognition model with the minimum deviation when the difference value between the deviation of the training recognition model with the minimum deviation and the deviation of the fatigue recognition model reaches a preset deviation threshold value.
5. The fatigue driving monitoring method based on cloud intelligent learning of claim 1, wherein the method is characterized by sending out an early warning based on the fatigue degree and sending the fatigue degree to the cloud database, and specifically comprises: according to the fatigue degree, matching a preset fatigue early warning table to determine an early warning mode; When the fatigue degree is mild fatigue, reminding the driver through acousto-optic early warning; and when the fatigue degree is deep fatigue, reporting driver information to a vehicle supervision platform, and gradually reducing the vehicle speed to stop and opening the double flashing of the vehicle.
6. The cloud intelligent learning-based fatigue driving monitoring method according to claim 1, wherein the processing the weighting factor of each neuron based on a preset updating algorithm to update the fatigue recognition model specifically comprises: Marking the factor position of the weight factor of each neuron, wherein the factor position is the corresponding position of the weight factor of each neuron in the training recognition model with the minimum deviation; And inquiring the neuron position which is the same as the factor position in the fatigue identification model, and replacing the weight factor of each neuron with the corresponding one in the fatigue identification model so as to update the fatigue identification model.
7. Fatigue driving monitoring equipment based on high in clouds intelligence study, its characterized in that, equipment includes: At least one processor; And a memory communicatively coupled to the at least one processor; Wherein the memory stores instructions executable by the at least one processor, the instructions are executable by the at least one processor to enable the at least one processor to: the method comprises the steps of obtaining monitoring video data of a driver and sending the monitoring video data to a cloud, wherein the monitoring video data is a video comprising a face and a head of the driver, and the cloud comprises a cloud database and a cloud computing module, and the cloud database is used for storing the monitoring video data; processing the monitoring video data based on a preset image preprocessing algorithm to extract a plurality of facial features of the driver, wherein the plurality of facial features comprise a yawning frequency, a blinking frequency, eyelid closing time, pupil constriction amount, sight line offset out-of-bounds and sight line jump out-of-bounds frequency; Inputting the plurality of facial features as feature parameters into a fatigue recognition model; processing the plurality of facial features based on a hidden layer of the fatigue recognition model to determine a first weight of the plurality of facial features, wherein the first weight is related to a time at which the plurality of facial features appear; matching a preset weight matching table based on the facial features to obtain second weights of different facial features; multiplying the plurality of facial feature first weights by the second weight of the different facial feature to calculate a fatigue value for the driver; comparing the fatigue value of the driver with a preset multi-stage fatigue threshold value to determine the fatigue degree of the driver; Sending out early warning based on the fatigue degree, and sending the fatigue degree to the cloud database; Processing the cloud database based on the cloud computing module, and iteratively computing a training recognition model preset in the cloud computing module to obtain deviation of the training recognition model and output a training recognition model with minimum deviation, wherein the training recognition model and the fatigue recognition model are the same type of neural network model; processing the training recognition model with the minimum deviation based on a preset reverse operation algorithm to obtain a weight factor of each neuron of the training recognition model with the minimum deviation; And processing the weight factor of each neuron based on a preset updating algorithm to update the fatigue identification model.
8. A non-volatile computer storage medium for fatigue driving monitoring based on cloud intelligent learning, storing computer executable instructions, characterized in that the computer executable instructions are configured to: the method comprises the steps of obtaining monitoring video data of a driver and sending the monitoring video data to a cloud, wherein the monitoring video data is a video comprising a face and a head of the driver, and the cloud comprises a cloud database and a cloud computing module, and the cloud database is used for storing the monitoring video data; the method comprises the steps of obtaining monitoring video data of a driver and sending the monitoring video data to a cloud, wherein the monitoring video data is a video comprising a face and a head of the driver, and the cloud comprises a cloud database and a cloud computing module, and the cloud database is used for storing the monitoring video data; processing the monitoring video data based on a preset image preprocessing algorithm to extract a plurality of facial features of the driver, wherein the plurality of facial features comprise a yawning frequency, a blinking frequency, eyelid closing time, pupil constriction amount, sight line offset out-of-bounds and sight line jump out-of-bounds frequency; Inputting the plurality of facial features as feature parameters into a fatigue recognition model; processing the plurality of facial features based on a hidden layer of the fatigue recognition model to determine a first weight of the plurality of facial features, wherein the first weight is related to a time at which the plurality of facial features appear; matching a preset weight matching table based on the facial features to obtain second weights of different facial features; multiplying the plurality of facial feature first weights by the second weight of the different facial feature to calculate a fatigue value for the driver; comparing the fatigue value of the driver with a preset multi-stage fatigue threshold value to determine the fatigue degree of the driver; Sending out early warning based on the fatigue degree, and sending the fatigue degree to the cloud database; Processing the cloud database based on the cloud computing module, and iteratively computing a training recognition model preset in the cloud computing module to obtain deviation of the training recognition model and output a training recognition model with minimum deviation, wherein the training recognition model and the fatigue recognition model are the same type of neural network model; processing the training recognition model with the minimum deviation based on a preset reverse operation algorithm to obtain a weight factor of each neuron of the training recognition model with the minimum deviation; And processing the weight factor of each neuron based on a preset updating algorithm to update the fatigue identification model.

Description

Fatigue driving monitoring method, equipment and medium based on cloud intelligent learning Technical Field The application relates to the technical field of automobile monitoring, in particular to a fatigue driving monitoring method, equipment and medium based on cloud intelligent learning. Background Fatigue driving is a phenomenon in which a driver produces physiological and psychological disturbances after continuous driving for a long period of time, and driving skill is objectively reduced. The driver has poor or insufficient sleep quality, and is easy to fatigue after driving the vehicle for a long time. Fatigue in driving affects the attention, feel, perception, thinking, judgment, mind, decision, and movement of the driver. The fatigue driving occurs road traffic accident. Therefore, the driving of the vehicle is strictly prohibited after fatigue. In the prior art, for monitoring, researching and applying fatigue driving, the following aspects are mainly included: Firstly, in the aspect of fatigue driving external expression, by monitoring the mental state of a person, for example, physiological or action change conditions such as nervous system functions, circulation functions, blood indexes, eye parameters, breathing functions, body temperature fluctuation, head or face, driving operation behaviors and the like, subjective or involuntary reflex is formed, and the judgment and decision are realized by combining the monitoring technology, the monitoring method and the evaluation standard of the indexes. However, such monitoring devices are usually wearable devices, which have a large interference to the driver and are easy to image the normal driving of the driver. Secondly, in terms of vehicle parameters, such as that the automobile frequently exceeds a central line, the speed of the automobile is too fast or slow or is uncoordinated with the surrounding driving environment, abnormal rotation moment of the steering wheel occurs, and the like. The method has the advantages of strong real-time performance and no interference to the driver, but has the defects of being easily limited by the type of the automobile, and being difficult to formulate a unified evaluation standard due to individual difference, so that the accuracy of fatigue driving judgment is not high. Thirdly, in the aspect of complex intelligent operation, on one hand, the cost is too high, on the other hand, the calculation power of the vehicle-mounted controller is limited, the real-time requirement cannot be met, and a special sensor is needed, so that the vehicle-mounted controller is difficult to popularize in the aspect of automobile operation. Fourth, in terms of machine vision, the current general fatigue monitoring method based on machine vision can monitor the changes of the head and face states of the driver, such as frequent nodding or long-term immobility of the head, pupil diameter shrinkage, eyelid closure, slow blink rate and the like, for analysis and judgment when the driver is in a driving fatigue state. Problems are information dispersion, simple analysis strategy, difficult determination of decision criteria, poor accuracy, and susceptibility to experience and environmental changes, etc. Therefore, how to improve the accuracy, popularity and stability of monitoring fatigue driving without affecting the normal driving of the driver is a urgent problem to be solved. Disclosure of Invention The embodiment of the application provides a fatigue driving monitoring method, equipment and medium based on cloud intelligent learning, which are used for solving the following technical problems of improving the accuracy, popularity and stability of monitoring fatigue driving and not affecting the normal running of a driver. The embodiment of the application provides a fatigue driving monitoring method based on cloud intelligent learning, which is characterized by comprising the steps of obtaining monitoring video data of a driver and sending the monitoring video data to the cloud, wherein the monitoring video data comprise a video of the face and the head of the driver, the cloud comprises a cloud database and a cloud computing module, the cloud database is used for storing the monitoring video data, the monitoring video data are processed based on a preset image preprocessing algorithm to extract a plurality of facial features of the driver, the plurality of facial features comprise a yawing frequency, a blink frequency, eyelid closing time, pupil narrowing, line-of-sight deviation out-of-range and line-of-sight jump out-of-range frequency, the plurality of facial features are processed based on a fatigue recognition model to determine the fatigue degree of the driver, early warning is sent based on the fatigue degree and sent to the cloud database, the cloud computing module is used for processing the cloud database based on the fatigue degree to iterate a training recognition model preset in the cloud computing module to ob