CN-121287147-B - Driving fatigue monitoring auxiliary system based on Beidou satellite positioning and multi-mode data fusion technology

Abstract

The invention provides a driving fatigue monitoring auxiliary system based on Beidou satellite positioning and multi-modal data fusion technology, which relates to the field of electric digital data processing and comprises a multi-modal information sensing and collecting module, a data preprocessing and quality assurance module, a fatigue state recognition and assessment module and an intelligent early warning and self-adaptive optimization module, wherein the multi-modal information sensing and collecting module is responsible for acquiring the state, driving behavior and environment information of a driver in real time, the data preprocessing and quality assurance module is responsible for carrying out time-space alignment, quality assessment and feature standardization on multi-source data, the fatigue state recognition and assessment module is responsible for fusing multi-dimensional features and judging fatigue grade and risk trend, and the intelligent early warning and self-adaptive optimization module is responsible for implementing hierarchical intervention and continuously optimizing system performance.

Inventors

• YUAN DA
• LIU YUELIN
• WANG JINLIANG
• CAI RONG
• SONG JIHUA
• CAO WEI
• ZHANG TIEJUN
• FAN YU
• LI CHUHAN
• WANG YING
• XIAO MINGTAO

Assignees

• 湖南汽车工程职业学院
• 湖南创信伟立科技股份有限公司

Dates

Publication Date

20260512

Application Date

20251118

Claims (3)

1. 1. The driving fatigue monitoring auxiliary system based on the Beidou satellite positioning and multi-mode data fusion technology is characterized by comprising a multi-mode information sensing and collecting module, a data preprocessing and quality guaranteeing module, a fatigue state identifying and evaluating module and an intelligent early warning and self-adaptive optimizing module; The multi-mode information sensing and collecting module is responsible for acquiring the state, driving behavior and environmental information of a driver in real time, the data preprocessing and quality assurance module is responsible for carrying out space-time alignment, quality assessment and feature standardization on multi-source data, the fatigue state recognition and assessment module is responsible for fusing multi-dimensional features and judging fatigue grade and risk trend, and the intelligent early warning and self-adaptive optimization module is responsible for implementing hierarchical intervention and continuously optimizing system performance; The multi-mode information sensing and collecting module comprises a driver physiological state monitoring unit, a driving behavior feature extracting unit and an environment and vehicle state sensing unit, wherein the driver physiological state monitoring unit is used for collecting physiological and visual features in real time, the driving behavior feature extracting unit is used for monitoring driving behavior indexes through a vehicle-mounted sensing system and extracting behavior features, and the environment and vehicle state sensing unit is used for acquiring the geographic position, the running speed, the acceleration and the route track of a vehicle; The data preprocessing and quality guaranteeing module comprises a multi-source data space-time synchronization unit, a data quality evaluation and abnormality detection unit and a feature extraction and standardization unit, wherein the multi-source data space-time synchronization unit performs time synchronization and space coordinate calibration on data information from different sensors by taking Beidou satellite positioning time stamps as unified references, the data quality evaluation and abnormality detection unit performs integrity check and reliability evaluation on original data acquired by the sensors, and the feature extraction and standardization unit converts preprocessed multi-mode data into unified feature vector space; The fatigue state identification and assessment module comprises a multi-mode feature fusion calculation unit, a fatigue grade classification and judgment unit and a fatigue trend prediction and risk assessment unit, wherein the multi-mode feature fusion calculation unit is used for carrying out feature grade fusion on standardized physiological features, visual features and driving behavior features, the fatigue grade classification and judgment unit is used for classifying the current state of a driver into three grades of wakefulness, mild fatigue and severe fatigue, and the fatigue trend prediction and risk assessment unit is used for predicting fatigue evolution trend in a certain time window in the future; the intelligent early warning and self-adaptive optimization module comprises a grading early warning and multi-mode intervention unit, a personalized threshold self-adaptive unit and a system self-learning and performance optimization unit, wherein the grading early warning and multi-mode intervention unit implements differential warning in a multi-channel mode according to a fatigue grade discrimination result, the personalized threshold self-adaptive unit establishes a dynamic fatigue threshold model based on individual difference and historical behavior data of a driver, and the system self-learning and performance optimization unit carries out whole-course recording and effect evaluation on fatigue events, intervention measures and driver responses; The multi-modal feature fusion calculation unit comprises a feature level fusion processor, an attention mechanism processor and a risk index calculator, wherein the feature level fusion processor is used for inputting standardized physiological feature vectors, visual feature vectors and driving behavior feature vectors into a shared characterization learning network, extracting high-order semantic features of each mode through multi-layer nonlinear transformation, and realizing deep fusion of cross-modal information in a feature space, the attention mechanism processor is used for dynamically distributing weights of different mode features in a fusion process, strengthening a feature channel strongly related to a fatigue state, inhibiting interference of noise and redundant information, mining time sequence dependency and complementarity between multi-modal data, and the risk index calculator inputs the fused comprehensive feature vectors into a full-connection layer and an activation function and outputs continuous comprehensive fatigue risk index through regression calculation; The attention mechanism processor calculates fatigue gradient attention scores of all modes at the moment t according to the following formula: ; Wherein Q (t) is the query vector at the current time t, K i (t) is the key vector of the ith modality, d k is the dimensions of the query vector and the key vector, softmax () is the normalization function, For the gradient modulation of the intensity coefficient, For the time gradient of the fatigue risk index at time t, F i (t) refers to the eigenvector of the i-th modality, Is the mean value characteristic vector of three modes; The feature level fusion processor performs weighted fusion on the feature vectors of the three modes based on the attention scores to obtain a fusion feature vector F fus (t).
2. 2. The driving fatigue monitoring auxiliary system based on the Beidou satellite positioning and multi-mode data fusion technology of claim 1, wherein the attention mechanism processor calculates a cooperative gain coefficient between two modes according to the following formula: ; Where MI (F i ,F j |s) represents mutual information of the feature vectors of the ith mode and the jth mode under the condition of the state s, s ftg represents a fatigue state, s awk represents an awake state, The cross covariance of feature vectors representing the ith and jth modes within the current time window, Representing the self-variance of the ith modality feature vector.
3. 3. The driving fatigue monitoring auxiliary system based on the Beidou satellite positioning and multi-mode data fusion technology as set forth in claim 2, wherein the risk index calculator calculates the comprehensive fatigue risk index r (t) according to the following formula: ; Wherein, the For the sigmoid activation function, r ins (t) is the transient risk term, r tem (t) is the timing risk term, In order to trade-off the coefficients, For the co-gain modulation of the intensity coefficient, Is a cooperative gain modulation function; The instantaneous risk item is mapped according to the fusion feature vector, the time sequence risk item is obtained according to the change condition of the historical risk index, and the collaborative gain modulation function is calculated according to the following formula: ; Where M is the number of modes and w ij is the weight of the pair of modes (i, j).

Description

Driving fatigue monitoring auxiliary system based on Beidou satellite positioning and multi-mode data fusion technology Technical Field The invention relates to the field of electric digital data processing, in particular to a driving fatigue monitoring auxiliary system based on Beidou satellite positioning and multi-mode data fusion technology. Background Driving fatigue is one of the main reasons for traffic accidents, and the fatigue state is very easy to cause serious traffic accidents due to the fact that the response time of drivers is prolonged, the attention is reduced, and the judging capability is weakened. In order to effectively prevent fatigue driving, researchers develop a great deal of driving fatigue monitoring technical researches, and mainly comprise detection methods based on physiological signal detection, facial visual feature detection, driving behavior monitoring and multi-mode data fusion. The detection method based on the physiological signals judges the fatigue state by collecting physiological parameters such as an electroencephalogram signal, an electrocardiosignal, an electromyogram signal and the like of a driver. The method has higher judgment accuracy, but needs to adopt a contact sensor to cause uncomfortable feeling and interference to a driver, and has larger limitation in practical application. The detection method based on the facial visual features captures the facial image of the driver by using a camera, and judges the fatigue state by analyzing the eyelid closing degree, blink frequency, yawning times and other features. The method belongs to non-contact measurement, but is easily influenced by factors such as illumination change, glasses wearing of a driver and the like, and the detection accuracy is unstable. The detection method based on driving behavior indirectly deduces the fatigue state of the driver by monitoring the vehicle operation parameters such as steering wheel rotation angle, lane departure, vehicle speed change and the like, but the method is easily interfered by factors such as road conditions, driving habits and the like, and has lower reliability. Three fatigue detection algorithms commonly used in current research include a PERCLOS algorithm based on ocular features, an HRV analysis algorithm based on heart rate variability, and a deep learning algorithm based on multi-modal feature fusion. The PERCLOS algorithm evaluates the fatigue degree by calculating the eyelid closing time ratio in unit time, is simple to calculate and good in real-time performance, but only depends on single eye characteristics, the detection accuracy is obviously reduced when the illumination is insufficient or the driver wears glasses, and early signs of fatigue cannot be captured. The HRV analysis algorithm evaluates the active state of the autonomic nervous system by extracting the time domain and frequency domain characteristics of the electrocardiosignal, can reflect the fatigue change of a physiological layer, but needs a wearable sensor to acquire data, so that the driver is restrained, and the heart rate change is influenced by various factors such as emotion, movement and the like, and has insufficient specificity. The multi-modal fusion algorithm based on deep learning inputs multi-source data such as physiological signals, facial images and driving behaviors into a convolutional neural network or a cyclic neural network to perform feature extraction and fusion, so that the detection accuracy can be improved, but most of the existing methods adopt a simple feature splicing strategy, complementarity and time sequence relativity between different modalities cannot be fully mined, and the model training needs a large number of labeling samples, so that the calculation complexity is high, and the real-time requirement is difficult to meet. The foreign patent US8022831B1 discloses an interactive fatigue management system which provides rest site information to a driver by detecting fatigue and drowsiness states of the driver, and contacts friends or family of the driver through a mobile phone. The main disadvantage of the patent is that the fatigue detection method is single and only depends on face monitoring or driving behavior analysis, and the detection accuracy and the robustness are insufficient due to the lack of a multi-mode data fusion mechanism. After the system detects fatigue, the system mainly adopts information prompt and external contact ways to intervene, and cannot implement grading early warning according to the severity of the fatigue, so that the intervention strategy lacks pertinence. In addition, the patent does not relate to a space-time data synchronization technology based on Beidou satellite positioning, and cannot provide a unified time reference for multi-source sensor data, so that the problem of time sequence dislocation exists when different mode data are fused, and the real-time performance and accuracy of fatigue judgment are affected