CN-117547284-B - Driver state online identification system and method based on virtual reality

CN117547284BCN 117547284 BCN117547284 BCN 117547284BCN-117547284-B

Abstract

The invention discloses a driver state online identification system and method based on virtual reality, comprising the following steps of 1, building a virtual reality driving system by using a Unity 3D engine and virtual reality equipment, 2, acquiring original electroencephalogram signals and driving behavior data of a driver in a virtual reality environment by using an electroencephalogram acquisition equipment and the Unity 3D engine, 3, preprocessing the acquired data, 4, dividing different alertness states according to the preprocessed driving behavior data through a clustering K-means algorithm, extracting electroencephalogram signal time-frequency characteristics and variation coefficient characteristics in the different alertness states to obtain electroencephalogram data after extracting the characteristics, 5, training by using a machine learning algorithm to obtain a classification model with high classification accuracy, and 6, analyzing and processing the electroencephalogram signal data acquired in real time and then putting the electroencephalogram signal data into the classification model to obtain the driver alertness state of online identification. The invention can realize on-line identification and has high identification accuracy.

Inventors

HAN CHUNXIAO
GUO FENGJUAN
CHE YANQIU
QIN YINGMEI
LI SHANSHAN
QIN QING

Assignees

天津职业技术师范大学(中国职业培训指导教师进修中心)

Dates

Publication Date: 20260505
Application Date: 20220318

Claims (11)

1. The driver state online identification method based on virtual reality is characterized by comprising the following steps of: Step 1, a Unity 3D engine and virtual reality equipment are used for building a virtual reality driving system; step2, acquiring original electroencephalogram signals and driving behavior data of a driver in the virtual reality environment in the step1 by using electroencephalogram acquisition equipment and a Unity 3D engine; Step 3, preprocessing the original electroencephalogram signals and driving behavior data acquired in the step 2, and improving the credibility of the data; Step 4, dividing different alertness states through a clustering K-means algorithm according to the preprocessed driving behavior data, extracting time-frequency characteristics and variation coefficient characteristics of the electroencephalogram signals in the different alertness states, and obtaining electroencephalogram data after the characteristics are extracted; step 5, training the electroencephalogram data after extracting the features by using a machine learning algorithm to obtain a classification model with high classification accuracy; step 6, analyzing and processing the electroencephalogram signal data acquired in real time, and then putting the electroencephalogram signal data into a classification model to obtain the driver alertness state of online identification; in the step 1, the method for constructing the virtual reality driving system comprises the following steps: s111, building a complete automobile driving environment by using a Unity 3D engine; S112, rendering and splicing the road model through an Injector panel to form a straight road model, adding straight marks and pavement marking lines on the road, and screening and placing environmental scenes at two sides of the road; S113, creating a weather system by using the Unity 3D sky box; S114, the experimental scene is a virtual reality simulation driving environment, a tested person wears an electroencephalogram acquisition device to carry out driving tasks, and the alertness state of the tested person is displayed in real time at the upper right corner of the interface through a GUI display interface of a simulation driving system; s115, connecting a steering wheel in the simulated driving equipment with a computer through a USB interface, and controlling the running direction of the simulated vehicle by using the steering wheel; S116, the virtual reality device is connected with a computer through a streaming box, the two base stations capture the motion of the head display device, the motion information of the head display device is transmitted to Unity 3D, and then the Steam VR switches the real-time visible picture of human eyes according to the head rotation information and outputs the picture to the head display device, so that the 3D stereoscopic display of a scene is realized; In the step 4, the K-means algorithm clusters driving behavior data, and divides the state alertness of the driver into three categories, namely high alertness, half alertness and low alertness, wherein the clustering process is as follows: Wherein, the For each data point in the driving behaviour data, The number of data points is a number of data points, For the number of the cluster centers, Is that Clustering centers after the secondary iteration; In the step 4, d 3 sub-frequency bands are selected for extracting the time-frequency characteristics of the electroencephalogram signals, and when the characteristics of the variation coefficients are extracted, Wherein the method comprises the steps of The standard deviation of the d 3 subband representing samples; representing the average value of d 3 sub-bands of the sample, wherein CV is the electroencephalogram data after the characteristics are extracted; In the step 5, a support vector machine algorithm is utilized, and the electroencephalogram data after feature extraction is adopted to train a classification model: Wherein, the In order to train the sample, For the number of samples, w and b are respectively the normal vector and intercept of the hyperplane constructed by the algorithm of the support vector machine.
2. The method for on-line recognition of driver state based on virtual reality according to claim 1, wherein in the step 2, the electroencephalogram acquisition device acquires original electroencephalogram signals at four leads of P8, T7, T8 and P7, and the driving behavior data includes real-time, vehicle left-right offset and steering wheel angle.
3. The virtual reality-based driver state online recognition method of claim 2, wherein the sampling frequency of the original electroencephalogram signal is 500Hz.
4. The virtual reality-based driver state online identification method of claim 2, wherein the driving behavior data sampling rate is 30Hz.
5. The virtual reality-based driver state online identification method according to claim 1 is characterized in that in the step 3, the preprocessing method of the original electroencephalogram signals comprises the steps of carrying out band-pass filtering in a range of 0.5 Hz-40 Hz, reducing the sampling rate from 500Hz to 256Hz, selecting electroencephalogram data with fixed duration for subsequent analysis, and obtaining electroencephalogram data of m samples after the electroencephalogram data are processed through a data sliding window with fixed window length and step length.
6. The virtual reality-based driver state online identification method of claim 5, wherein the fixed duration is 60 minutes.
7. The virtual reality-based driver state online identification method of claim 5, wherein m = 1200.
8. The virtual reality-based driver state online identification method according to claim 1, wherein in the step 3, the driving behavior data preprocessing method comprises the steps of selecting driving behavior data with fixed duration for subsequent analysis, wherein the driving behavior data corresponds to each data segment of electroencephalogram data one by one, the driving behavior data sets window length and step length of a sliding window to be identical with the electroencephalogram data, and average values of left and right offset of a vehicle and angle of a steering wheel in each window of the driving behavior are calculated as statistical indexes, so that driving behavior variables of M samples are finally obtained.
9. The virtual reality-based driver state online identification method of claim 8, wherein M = 1200.
10. The virtual reality-based driver state online identification method according to claim 1, wherein in the step 6, the analysis processing method comprises the steps of carrying out band-pass filtering in a range of 0.5 Hz-40 Hz, reducing the sampling rate from 500Hz to 256Hz, then carrying out discrete wavelet transformation, selecting a d 3 sub-band, and extracting CV features through variation coefficients; driver alertness states include high alertness, semi-alertness, and low alertness.
11. The driver state online identification system based on virtual reality is characterized by comprising an electroencephalogram acquisition device, an analysis module and a classification model obtained by the method according to any one of claims 1-10, wherein the electroencephalogram acquisition device acquires original electroencephalogram signals and wirelessly transmits signals in real time through a local area network, the analysis module receives the original electroencephalogram signals, performs preprocessing filtering downsampling, extracts alertness related characteristics, and analyzes the characteristics by the classification model to obtain the driver alertness state.

Description

Driver state online identification system and method based on virtual reality Technical Field The invention relates to the technical field of driver alertness state online identification, in particular to a driver state online identification system and method based on virtual reality. Background The on-line recognition system for the driver alertness state is highly valued in various countries because of the development prospect of the traffic accident prevention, researchers can conduct various researches according to the characteristics of physiology and operation when the driver is tired, and the detection method for the driver alertness state can be roughly divided into detection methods based on the physiological signals of the driver, the driving behavior characteristics of the driver, the response time of the driver and the facial expression characteristics. The driver state is identified based on physiological signal features, mainly by analyzing the acquired signals of the brain electricity, the electrocardio, the electrooculogram and the like of the driver. Electroencephalogram signals have been considered as "gold standards" for detecting changes in alertness, and driver alertness states are detected by analyzing changes in delta, theta, alpha, beta waves in the electroencephalogram signals. The research shows that when the alertness is reduced, the activity of the corresponding low-frequency signal in the brain electrical signal is increased, and the amplitude of the event-related potential is correspondingly reduced. The driver state is identified based on the driving behavior features, and is mainly analyzed according to the parameter change of the vehicle during the driving process of the driver. When the driver's alertness is lowered, the sensing ability to the outside is lowered, and when the vehicle is deviated, the deviation of the vehicle cannot be adjusted in time due to the delay of the driver's operation of the steering wheel. Therefore, the control accuracy of the driver on the vehicle is reduced, the vehicle state parameter is greatly changed, and the driver state can be estimated by researching the vehicle state parameter change. The researchers at home and abroad study the response time of the driver based on the reduction of the perception ability of the driver in the fatigue state. The response time of each subject is different, and is inherent to each person, and the stress response of different persons to the same event is different. Therefore, the rule between the response time of each driver and the electroencephalogram signal of each driver needs to be studied to carry out overall judgment on the vigilance state of the driver. The driver state recognition research based on facial expression features is often carried out by using an image processing technology to judge, and a vehicle-mounted camera is used for collecting facial expression videos of a driver in the driving process and analyzing the facial expression videos in an offline state so as to judge the state of the driver at the moment. Analysis is performed in an off-line state, reminding cannot be performed in real time, and hysteresis is achieved. Disclosure of Invention The invention aims at solving the technical defects existing in the prior art and provides a driver state online identification method based on virtual reality. Another object of the present invention is to provide an on-line driver status recognition system based on the recognition method. The technical scheme adopted for realizing the purpose of the invention is as follows: A driver state online identification method based on virtual reality comprises the following steps: Step 1, a Unity 3D engine and virtual reality equipment are used for building a virtual reality driving system; step2, acquiring original electroencephalogram signals and driving behavior data of a driver in the virtual reality environment in the step1 by using electroencephalogram acquisition equipment and a Unity 3D engine; Step 3, preprocessing the original electroencephalogram signals and driving behavior data acquired in the step 2, and improving the credibility of the data; Step 4, dividing different alertness states through a clustering K-means algorithm according to the preprocessed driving behavior data, extracting time-frequency characteristics and variation coefficient characteristics of the electroencephalogram signals in the different alertness states, and obtaining electroencephalogram data after the characteristics are extracted; step 5, training the electroencephalogram data after extracting the features by using a machine learning algorithm to obtain a classification model with high classification accuracy; and 6, analyzing and processing the electroencephalogram signal data acquired in real time, and then putting the electroencephalogram signal data into a classification model to obtain the driver alertness state of online identification. In the above technica