CN-115952440-B - Motor imagery task classification method based on 3D interpolation and 3DCNN

Abstract

The invention discloses a motor imagery task classification method based on 3D interpolation and 3DCNN, which comprises the steps of firstly carrying out band-pass filtering processing on motor imagery electroencephalogram signals, then carrying out frequency domain transformation on EEG signals of each electrode by utilizing Fast Fourier Transformation (FFT) and obtaining power values, then projecting 3D coordinates of scalp electrodes into a 3D space, interpolating the power values by using a 3D interpolation algorithm to generate a 3D interpolation characteristic image containing 3D real space position information of the electrodes, and finally designing a 3D convolutional neural network (3 DCNN) to match the characteristics of the 3D interpolation characteristic image for characteristic extraction and classification. The invention embodies depth information of motor imagery activation, encodes accurate three-dimensional space information of the electrode into a 3D interpolation imaging diagram, and well matches the space convolution capability of 3 DCNN.

Inventors

• LI MINGAI
• ZHANG JING
• SUN YANJUN

Assignees

• 北京工业大学

Dates

Publication Date

20260505

Application Date

20230104

Claims (1)

1. 1. The motor imagery task classification method based on the 3D interpolation and the 3DCNN is characterized in that: firstly, carrying out band-pass filtering treatment on an original motor imagery electroencephalogram signal MI-EEG, then carrying out frequency domain transformation on the EEG signal of each electrode, solving a power value, generating a three-dimensional interpolation imaging diagram through a three-dimensional interpolation algorithm by combining with accurate three-dimensional position information of the electrode, and splicing the three-dimensional interpolation imaging diagram into a feature matrix; The method is characterized in that: Step1 pretreatment of EEG signals; carrying out 8-32Hz band-pass filtering on the electroencephalogram signals to obtain electroencephalogram signals most relevant to motor imagery tasks; Step2, obtaining a power value based on fast Fourier transform FFT; step2.1, extracting time-frequency characteristics by using a Fast Fourier Transform (FFT); The MI-EEG signal for the mth lead in a single acquisition experiment, where m represents the lead number, m e {1,2,3,.. The number of leads is N c }, N s represents the number of sampling points the single acquisition experiment contains, and the MI-EEG data for the mth lead can be expressed as: X m is then divided into N D windows, N D ∈N + , the data for each window can be expressed as: Where j is the window sequence number, j e {1,2,3,., N D }, so The number of the contained sampling points is as follows: In order to improve the computational resolution of the frequency, the signal sequence of each window is zero-padded to a length N FFT , and the transformed sequence is expressed as: furthermore, the 8-30Hz band sequence corresponding to the FOI is expressed as: the length of the band sequence, N F , is solved by: wherein F H and F L represent an upper frequency limit and a lower frequency limit of each sub-band, and F s is a sampling frequency; Step2.2, the power value for each electrode is calculated, and for each window of the band sequence, the average power is calculated independently as follows: from the average power values in N D windows, the solution is given by: Step3, generating a three-dimensional interpolation imaging diagram; The three-dimensional grid of the electrodes is obtained from an electrode distribution diagram of a BCI system, and the three-dimensional grid with the size of 32 multiplied by 32 is built according to the three-dimensional space position coordinates of the electrodes; Step3.2, interpolating the power value according to the 3D space position of the electrode and the established grid, expanding the time-frequency characteristic to a three-dimensional space, and generating a three-dimensional interpolation imaging diagram; step4, cascading 3DCNN recognition feature matrixes by four modules; The three-dimensional interpolation imaging method is characterized in that a three-dimensional interpolation imaging diagram is adopted in a step4.1, 3DCNN (direct sequence number) of four module cascade structures is designed and used for extracting and classifying feature matrixes, a four module cascade structure is adopted, a network can effectively decode coding information of a motor imagery task, the structures of the module 1 and the module 2 are the same, the three-dimensional interpolation imaging diagram comprises two 3D convolution layers and a maximum pooling layer, convolution kernels with the same size are 3X 3 and step sizes 3X 3, the activation functions of the two convolution layers are all ReLU, the module 3 comprises one 3D convolution layer and one maximum pooling layer, the convolution kernels are 3X 3, the step sizes are 3X 3, the activation functions of the convolution layers are ReLU, the module 4 comprises two fully-connected layers, the extracted spatial features are flattened and the types are output, and in order to avoid the situation that the network is over-fitted and the network training process is accelerated, the four modules adopt a batch normalization technology and a Dropout technology; And 3DCNN is used for identifying the characteristic matrix of the three-dimensional interpolation imaging diagram by the step4.2, and the characteristic matrix of the three-dimensional interpolation imaging diagram obtained by each electroencephalogram experiment is spliced to form a total data set which is used for training and testing the 3DCNN to realize task classification of motor imagery.

Description

Motor imagery task classification method based on 3D interpolation and 3DCNN Technical Field The invention relates to a three-dimensional (3D) interpolation and three-dimensional convolutional neural network (3 DCNN) which is used in the technical field of motor imagery electroencephalogram (MI-EEG) signal identification. The method specifically relates to a method for generating a three-dimensional interpolation imaging image by combining accurate three-dimensional position information of electrodes and generating a three-dimensional interpolation imaging image by a three-dimensional interpolation algorithm, wherein the EEG signals of each lead are subjected to frequency domain transformation based on Fast Fourier Transformation (FFT), power values are obtained. Finally, a 3D convolutional neural network (3 DCNN) is designed to match the characteristics of the 3D interpolation characteristic image to perform characteristic extraction and identification, and classification of motor imagery electroencephalogram signals is achieved. Background The brain-computer interface (BCI) enables direct communication between the brain and external devices, completely independent of the participation of muscles and peripheral nerves. Non-invasive BCI systems are a subject of intense research due to their simplicity, low cost, and non-invasiveness to the subject. Motor imagery electroencephalogram (MI-EEG) signals collected based on electrode caps are typically used as input signals for non-invasive BCI systems. By decoding MI-EEG signals, the movement intention of the subject is directly converted into control signals for external devices, which are widely used in the fields of disability, rehabilitation medicine, entertainment, games, etc. The MI-EEG signal is a non-stationary signal with well-defined rhythmic features and different motor imagery tasks with different information in the three-dimensional space activated. With the development of deep learning technology, some researchers propose various MI-EEG two-dimensional (2D) imaging methods based on a frequency domain or time-frequency analysis method, and design a two-dimensional convolutional neural network (2 DCNN) to realize feature automatic extraction and recognition. The two-dimensional space information of the electrode is utilized, so that the decoding effect of the motor imagery is obviously improved. However, depth information of motor imagery activation is not yet embodied, accurate three-dimensional spatial information of the electrodes cannot be encoded into the MI-EEG imaging map, resulting in insufficient spatial information of the electrodes, affecting MI-EEG decoding accuracy. Disclosure of Invention Aiming at the defects, the invention provides a motor imagery task classification method based on 3D interpolation and 3 DCNN. (1) The frequency bands most relevant to motor imagery tasks are obtained by filtering the original MI-EEG for all channels using band pass filters. The signal of each lead is subjected to frequency domain transformation by Fast Fourier Transform (FFT) and a power value is found. (2) And establishing a three-dimensional grid through the real 3D space position coordinates of the electrode, and interpolating the power value according to the 3D space position of the electrode and the established grid. And expanding the frequency domain features to a three-dimensional space to generate a three-dimensional interpolation imaging diagram. The signature contains the real-space positional information of the electrodes. (3) Based on the characteristics of the feature matrix of the three-dimensional interpolation imaging diagram, the 3DCNN with four module cascade structures is designed and used for extracting the feature matrix and decoding the motor imagery task, so that the spatial convolution calculation of the 3DCNN is more consistent with the spatial distribution characteristic of the MI-EEG signal in a physical sense. The specific steps of the invention are as follows: Step1 pretreatment of EEG signals. And carrying out 8-32Hz band-pass filtering on the electroencephalogram signals to obtain the electroencephalogram signals most relevant to the motor imagery task. Step2 finds the power value based on a Fast Fourier Transform (FFT). Step2.1 uses a Fast Fourier Transform (FFT) to extract the frequency domain features.The MI-EEG signal for the mth lead in a single acquisition experiment, where m represents the lead number, m e {1,2,3,.. The number of leads is N c},Nc, and N s represents the number of sampling points that a single acquisition experiment contains. The MI-EEG data of the mth lead can be expressed as: Then, x m is divided into N D windows, and N D∈N+,N+ is a positive natural number. The data for each window can be expressed as: where j is the window sequence number, j e {1,2,3,..N D }. So that The number of the contained sampling points is as follows: the FFT is used to convert the time domain signal for each window to th