CN-121971095-A - Multi-scale prediction system for attention decay of train driver based on electroencephalogram signals

CN121971095ACN 121971095 ACN121971095 ACN 121971095ACN-121971095-A

Abstract

The application discloses a train driver attention decay multiscale prediction system based on an electroencephalogram signal, which relates to the technical field of intelligent driving safety monitoring and comprises a deep learning model LiteFormer-BR based on an original multichannel electroencephalogram signal, wherein the model adopts a light CNN-transducer backbone network to extract space-time characteristics, receives a fixed length characteristic vector output by the light hybrid CNN-transducer backbone network, predicts a head through probability regression based on Beta distribution, simultaneously predicts continuous attention scores corresponding to K preset key time points in the future, and outputs a multiscale attention prediction result to realize multiscale prediction of an attention decay state of the train driver.

Inventors

YU DONG
TANG XING
Niu Zhengbei
WANG JIN
ZHI JINYI

Assignees

西南交通大学

Dates

Publication Date: 20260505
Application Date: 20260126

Claims (10)

1. The train driver attention decay multi-scale prediction system based on the electroencephalogram signals is characterized by comprising an electroencephalogram signal acquisition module, a signal processing and feature encoding module, a multi-scale attention prediction module and a prediction result output module; The electroencephalogram signal acquisition module is used for acquiring original multichannel electroencephalogram signals of a train driver; the signal processing and feature encoding module is used for carrying out feature extraction on the original multichannel electroencephalogram signals and generating a fixed-length feature vector for attention prediction; The multiscale attention prediction module is used for receiving the fixed length feature vector output by the high-efficiency light backbone network in the signal processing and feature encoding module, predicting the head through probability regression based on Beta distribution, and simultaneously predicting continuous attention scores corresponding to K preset key time points in the future; and the prediction result output module is used for outputting the multi-scale attention prediction result so as to realize multi-time scale prediction of the attention decline state of the train driver.
2. The system for predicting the attention decay multiscale of the train driver based on the electroencephalogram signals according to claim 1, wherein the feature extraction comprises an end-to-end deep learning model LiteFormer-BR (1) which consists of a lightweight hybrid CNN-transporter backbone network (2) and a probability regression prediction head (11) based on Beta distribution, the lightweight hybrid CNN-transporter backbone network (2) adopts a framework combining depth separable convolution and a linear attention mechanism and is used for extracting space-time features from input original multichannel electroencephalogram signals and generating compact feature representations, and sequentially comprises a block embedding module (3), at least two CNN-to-transporter modules (6) and C2T2 modules (7) which are connected in series, a final bottleneck module (10) and a global average pooling layer GAP (9), and the probability regression prediction head (11) based on Beta distribution is used for receiving fixed-length feature vectors output by the backbone network and simultaneously predicting continuous attention scores of K preset key time points in the future.
3. The multi-scale prediction system for attention decay of a train driver based on an electroencephalogram signal according to claim 2 is characterized in that the block embedding module (3) comprises at least two parallel one-dimensional convolution branches which are respectively configured to be of different receptive field scales, wherein a first convolution branch adopts a convolution kernel of a smaller size and is used for capturing microscopic local features of the electroencephalogram signal, a second convolution branch adopts a convolution kernel of a larger size and is larger than the first convolution branch in size and is used for capturing macroscopic trend features of the electroencephalogram signal, and outputs of the two branches are spliced in a channel dimension to form a fused initial feature map.
4. The system for predicting the attention decay multiscale of the train driver based on the electroencephalogram signals according to claim 2 is characterized in that each CNN-to-converter module (6, 7) further comprises a lightweight bottleneck convolution block (8) and a separable multi-head self-attention mechanism SMHSA (9), the lightweight bottleneck convolution block (8) is formed by replacing standard convolution layers with 3×3 depth separable convolution layers (DEPTHWISE SEPARABLE CONV) and sequentially connecting with a ReLU activation function and a 1×1 point convolution layer in order to reduce the parameter and the calculation cost, the lightweight bottleneck convolution block (8) in the C2T1 module (6) downsamples an input feature map to 32×64×16, and the lightweight bottleneck convolution block (8) in the C2T2 module (7) keeps the spatial resolution unchanged.
5. The system for predicting the attention decay multiscale of the train driver based on the electroencephalogram signals, which is characterized in that the separable multi-head self-attention mechanism SMHSA (9) adopts a lightweight design, 8 attention heads are arranged, the calculation process is that input features are respectively subjected to linear projection to obtain query Q, key K and value V, and then the Softmax operation in standard dot product attention is replaced by element-by-element multiplication and normalization operation, so that the calculation complexity is reduced from O (n 2) to O (n), wherein n is the sequence length, and long-range dependence modeling with low resource consumption is realized.
6. The system for multi-scale prediction of attention deficit of train drivers based on electroencephalogram signals according to claim 2, wherein the final bottleneck module (10) is composed of a3×3 depth separable convolution layer, a ReLU activation function and a1×1 convolution layer which are sequentially connected, and is used for compressing a high-dimensional space-time feature map processed by the C2T2 module (7) into a 64×32×8 tensor, and then the tensor is subjected to an average operation along the time and channel dimensions through the global average pooling layer GAP (9) to output a 64-dimensional fixed-length feature vector.
7. The multi-scale prediction system for the attention decay of the train driver based on the electroencephalogram signals is characterized in that the probability regression prediction head (11) based on Beta distribution comprises an alpha parameter head (12) and a Beta parameter head (13) which are parallel to each other, the alpha parameter head (12) consists of two fully connected layers, a LeakyReLU activation function is adopted in the middle of the alpha parameter head (12) to finally output K neurons, the Beta parameter head (13) has the same structure as the alpha parameter head (12), and the final output of the two branches is enabled to be the function through Softplus.
8. The system for multi-scale prediction of attention deficit of a train driver based on electroencephalogram signals according to claim 7, wherein K is equal to 5 and corresponds to five preset time scales of an instant state T+0s, an approaching early warning T+20s, a short-term prediction T+2min, a medium-term trend T+10min and a long-term trend T+30min respectively, and the attention score of each time point is estimated through an average value alpha/(alpha+beta) of corresponding Beta distribution.
9. The system for multi-scale prediction of attention deficit of a train driver based on electroencephalogram signals according to claim 1, wherein the model training loss function uses a negative log likelihood loss of Beta distribution, and the expression is: ; Where N is the batch size, K is the number of predicted time points, yij is the true attention score of the ith sample at the jth time point, αij and βij are the corresponding Beta distribution parameters predicted by the model, and Γ (·) is a Gamma function.
10. The system for multi-scale prediction of attention deficit of a train driver based on electroencephalogram signals according to claim 1, wherein the electroencephalogram signal acquisition module further comprises an experimental data acquisition and state labeling sub-module for synchronously acquiring electroencephalogram signals and attention behavior response data in the train driving task process, wherein: The electroencephalogram signal acquisition module acquires electroencephalogram signals of a plurality of functional brain areas of the head of a train driver in real time by adopting a multichannel electroencephalogram acquisition mode conforming to the topological distribution of preset electroencephalogram electrodes, and outputs multichannel time-synchronous electroencephalogram data at a preset sampling frequency; the experimental data acquisition and state labeling sub-module is used for embedding an alertness response task in a driving task interface, generating visual stimulus in a non-fixed time interval and recording behavior response time of a driver so as to represent the attention state change of the driver; The experimental data acquisition and state labeling submodule constructs an attention state evaluation index based on the behavior response time, and accordingly performs attention state labeling on the electroencephalogram data in a corresponding time period; the multichannel electroencephalogram data after time synchronization and state labeling is transmitted to the signal processing and feature encoding module and is used as input data of a multi-scale attention prediction model.

Description

Multi-scale prediction system for attention decay of train driver based on electroencephalogram signals Technical Field The invention relates to the technical field of intelligent driving safety monitoring, in particular to a multi-scale prediction system for attention decay of a train driver based on an electroencephalogram signal. Background During high speed rail and urban rail transit operations, train drivers need to perform highly focused driving tasks in a relatively closed, monotonic, and long lasting environment. Is influenced by factors such as long-time supervision, circadian rhythm variation, running environment repeatability and the like, and a driver is liable to have state changes such as attention loss, slow response and the like. Once attention decay occurs, the response to signal changes and sudden conditions may be untimely, thus constituting a potential risk for driving safety. Therefore, how to continuously monitor the attention state of a train driver and recognize the attention decay trend in advance has become an important research direction in the field of intelligent driving safety monitoring. To overcome the hysteresis of behavior monitoring, objective detection techniques based on electroencephalogram (EEG) physiological signals have been developed. Although a certain progress has been made in constructing a machine learning classifier by using the power spectrum characteristics of a specific frequency band (such as Alpha wave and Theta wave), the technical problems that are not solved in the prior art scheme still exist in actual deployment, namely, the limitation of a modeling mode is first. In the existing method, discrete labels are mostly adopted for supervision training, and attention decay is ignored to be a continuous dynamic evolution process, so that a model is difficult to output a refined state quantization index. Second, the uniqueness of the scale is predicted. The existing system can only conduct state judgment at a single moment, lacks overall capability from second-level instantaneous early warning to minute-level trend prediction, and cannot adapt to different requirements of different safety strategies on time advance. Third, existing deep learning models (e.g., standard transformers or deep CNNs) typically have large parameter amounts and high computational complexity, belonging to heavy models. The on-board computing terminal resources of the train cab are often limited (calculation power, memory and power consumption), and the real-time operation of the high-load models is difficult to support, so that advanced algorithms cannot be really deployed on the ground. Fourth, the prior art generally only enables single point-in-time or instant identification. Although some studies have attempted to combine eye movement, heart rate variability or multimodal features to make fatigue predictions and achieve some success within a specific time window, there is no technology available that can simultaneously achieve high-precision, multi-scale continuous attention level predictions from the original EEG signal end-to-end and elucidate the dynamic neural precursor evolution laws behind it. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a multi-scale prediction system for the attention decay of a train driver based on an electroencephalogram signal, which solves the problems in the background art. The invention aims at realizing the technical scheme that the attention decline multi-scale prediction system of the train driver based on the electroencephalogram comprises an electroencephalogram signal acquisition module, a signal processing and feature encoding module, a multi-scale attention prediction module and a prediction result output module; The electroencephalogram signal acquisition module is used for acquiring original multichannel electroencephalogram signals of a train driver; the signal processing and feature encoding module is used for carrying out feature extraction on the original multichannel electroencephalogram signals and generating a fixed-length feature vector for attention prediction; The multiscale attention prediction module is used for receiving the fixed length feature vector output by the high-efficiency light backbone network in the signal processing and feature encoding module, predicting the head through probability regression based on Beta distribution, and simultaneously predicting continuous attention scores corresponding to K preset key time points in the future; and the prediction result output module is used for outputting the multi-scale attention prediction result so as to realize multi-time scale prediction of the attention decline state of the train driver. And the prediction result is used for driving an operation decision of a train driving safety monitoring or early warning system. Preferably, the feature extraction comprises an end-to-end deep learning model LiteFormer-BR (1), which consist