US-12622627-B2 - Electroencephalogram signal classification method and apparatus, device, storage medium and program product

Abstract

An electroencephalogram (EEG) signal classification method and apparatus, a device, a storage medium, and a program product are provided, and relate to the field of signal processing technologies. The method includes: obtaining a first EEG signal; obtaining time-frequency feature maps of at least two electrode signals in the first EEG signal; performing feature extraction based on the time-frequency feature maps of the at least two electrode signals to obtain a first extracted feature map; performing weighting processing based on an attention mechanism on the first extracted feature map to obtain an attention feature map; and obtaining a motor imagery type of the first EEG signal based on the attention feature map.

Inventors

• Luyan LIU
• Kai Ma
• Yefeng Zheng

Assignees

• TENCENT TECHNOLOGY (SHENZHEN) COMPANY LTD

Dates

Publication Date

20260512

Application Date

20221031

Priority Date

20210226

Claims (18)

1. 1 . An electroencephalogram (EEG) signal classification method, performed by at least one processor, the method comprising: obtaining a first EEG signal, the first EEG signal comprising at least two electrode signals, an electrode signal of the at least two electrode signals indicating an EEG signal generated by a target object in a spatial region corresponding to the electrode signal; obtaining time-frequency feature maps of the at least two electrode signals, a time-frequency feature map indicating a time-domain feature and a frequency-domain feature of the electrode signal; performing feature extraction based on the time-frequency feature maps of the at least two electrode signals through a first convolutional layer in an EEG signal classification model to obtain a first extracted feature map, the first extracted feature map being fused with spatial features of the at least two electrode signals, and the spatial features of the at least two electrode signals being related to spatial regions corresponding to the at least two electrode signals; performing, based on a first attention weighted network in the EEG signal classification model, weighting processing based on an attention mechanism on the first extracted feature map to obtain an attention feature map of the first EEG signal; and obtaining a motor imagery type of the first EEG signal based on the attention feature map of the first EEG signal, wherein the EEG signal classification model is trained based on a motor imagery type corresponding to a first sample EEG signal and a sample probability distribution corresponding to the first sample EEG signal, the sample probability distribution indicating probabilities that the first sample EEG signal is of each of various motor imagery types, respectively.
2. 2 . The method according to claim 1 , wherein the attention mechanism comprises at least one of a spatial attention mechanism or a channel attention mechanism.
3. 3 . The method according to claim 2 , wherein the first attention weighted network comprises a first spatial attention weighted network, a second convolutional layer, a first channel attention weighted network, and a third convolutional layer; and the performing, based on the first attention weighted network in the EEG signal classification model, the weighting processing based on the attention mechanism on the first extracted feature map to obtain the attention feature map of the first EEG signal comprises: performing, based on the first spatial attention weighted network, weighting processing based on the spatial attention mechanism on the first extracted feature map to obtain a first spatial feature map; performing feature extraction on the first spatial feature map based on the second convolutional layer to obtain a second extracted feature map; performing, based on the first channel attention weighted network, weighting processing based on the channel attention mechanism on the second extracted feature map to obtain a first channel feature map; performing feature extraction on the first channel feature map based on the third convolutional layer to obtain a third extracted feature map; and obtaining the attention feature map based on the first spatial feature map, the first channel feature map, and the third extracted feature map.
4. 4 . The method according to claim 3 , wherein the first attention weighted network further comprises a second attention weighted network; and the obtaining the attention feature map based on the first spatial feature map, the first channel feature map, and the third extracted feature map comprises: fusing the first spatial feature map, the first channel feature map, and the third extracted feature map to obtain a first fused feature map; and performing, through the second attention weighted network, weighting processing based on the attention mechanism on the first fused feature map to obtain the attention feature map.
5. 5 . The method according to claim 4 , wherein the second attention weighted network comprises a second spatial attention weighted network and a second channel attention weighted network; and the performing, through the second attention weighted network, the weighting processing based on the attention mechanism on the first fused feature map to obtain the attention feature map comprises: performing, through the second channel attention weighted network, weighting processing based on the channel attention mechanism on the first fused feature map to obtain a second channel feature map; and performing, through the second spatial attention weighted network, weighting processing based on the spatial attention mechanism on the second channel feature map to obtain the attention feature map.
6. 6 . The method according to claim 1 , wherein the obtaining the time-frequency feature maps of the at least two electrode signals comprises: performing continuous wavelet transform based on the at least two electrode signals to obtain the time-frequency feature maps of the at least two electrode signals.
7. 7 . The method according to claim 6 , wherein the EEG signal classification model further comprises a first fully connected layer; and the obtaining the motor imagery type of the first EEG signal based on the attention feature map of the first EEG signal comprises: performing, through the first fully connected layer, data processing on the attention feature map of the first EEG signal to obtain a feature vector of the first EEG signal; obtaining a probability distribution of the first EEG signal based on the feature vector of the first EEG signal, the probability distribution indicating probabilities that the first EEG signal is of each of a plurality of motor imagery types, respectively; and determining the motor imagery type of the first EEG signal based on the probability distribution of the first EEG signal.
8. 8 . An electroencephalogram (EEG) signal classification method, performed by at least one processor, the method comprising: obtaining a first sample EEG signal, the first sample EEG signal comprising at least two first sample electrode signals, and a first sample electrode signal indicating an EEG signal generated in a spatial region corresponding to the first sample electrode signal from a target object that performs motor imagery; obtaining first sample time-frequency feature maps of the at least two first sample electrode signals, a first sample time-frequency feature map indicating a time-domain feature and a frequency-domain feature of a corresponding first sample electrode signal; performing feature extraction on the first sample time-frequency feature maps of the at least two first sample electrode signals through a first convolutional layer in an EEG signal classification model, to obtain a first sample extracted feature map, the first sample extracted feature map being fused with spatial features of the at least two first sample electrode signals, and the spatial features of the at least two first sample electrode signals being related to spatial regions corresponding to the at least two first sample electrode signals; performing, based on an attention weighted network in the EEG signal classification model, weighting processing based on an attention mechanism on the first sample extracted feature map to obtain an attention feature map of the first sample EEG signal; obtaining a sample probability distribution of the first sample EEG signal based on the attention feature map of the first sample EEG signal, the sample probability distribution indicating probabilities that the first sample EEG signal is of each of a plurality of motor imagery types, respectively; and training the EEG signal classification model based on the sample probability distribution and a motor imagery type of the first sample EEG signal, the EEG signal classification model being configured to predict a motor imagery type of a first EEG signal.
9. 9 . The method according to claim 8 , wherein the method further comprises: obtaining a second sample EEG signal, the second sample EEG signal comprising at least two second sample electrode signals, and a second sample electrode signal indicating an EEG signal generated in a spatial region corresponding to the second sample electrode signal from a target object that performs motor imagery; obtaining second sample time-frequency feature maps of the at least two second sample electrode signals, a second sample time-frequency feature map indicating a time-domain feature and a frequency-domain feature of the second sample electrode signal; performing feature extraction on the second sample time-frequency feature maps of the at least two second sample electrode signals through the first convolutional layer in the EEG signal classification model, to obtain a second sample extracted feature map, the second sample extracted feature map being fused with spatial features of the at least two second sample electrode signals, and the spatial features of the at least two second sample electrode signals being related to spatial regions corresponding to the at least two second sample electrode signals; and performing, based on the attention weighted network in the EEG signal classification model, weighting processing based on the attention mechanism on the second sample extracted feature map to obtain the attention feature map of the second sample EEG signal; and the training the EEG signal classification model based on the sample probability distribution and the motor imagery type of the first sample EEG signal comprises: training the EEG signal classification model based on the sample probability distribution, the motor imagery type of the first sample EEG signal, the attention feature map of the first sample EEG signal, and the attention feature map of the second sample EEG signal.
10. 10 . The method according to claim 9 , wherein the second sample EEG signal indicates EEG signals generated by a target object corresponding to the first sample EEG signal at different moments; or the second sample EEG signal indicates an EEG signal generated by an object other than the target object corresponding to the first sample EEG signal.
11. 11 . An EEG signal classification apparatus, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code including: first signal obtaining code configured to cause the at least one processor to obtain a first EEG signal; the first EEG signal comprising at least two electrode signals, and an electrode signal indicating an EEG signal generated by a target object in a spatial region corresponding to the electrode signal; first time-frequency feature obtaining code configured to cause the at least one processor to obtain time-frequency feature maps of the at least two electrode signals, a time-frequency feature map indicating a time-domain feature and a frequency-domain feature of the electrode signal; first extracted feature obtaining code configured to cause the at least one processor to perform feature extraction based on the time-frequency feature maps of the at least two electrode signals through a first convolutional layer in an EEG signal classification model to obtain a first extracted feature map, the first extracted feature map being fused with spatial features of the at least two electrode signals, and the spatial features of the at least two electrode signals being related to spatial regions corresponding to the at least two electrode signals; first attention feature obtaining code configured to cause the at least one processor to perform, based on a first attention weighted network in the EEG signal classification model, weighting processing based on an attention mechanism on the first extracted feature map to obtain an attention feature map of the first EEG signal; and imagery type obtaining code configured to cause the at least one processor to obtain a motor imagery type of the first EEG signal based on the attention feature map of the first EEG signal, wherein the EEG signal classification model is trained based on a motor imagery type corresponding to a first sample EEG signal and a sample probability distribution corresponding to the first sample EEG signal, the sample probability distribution indicating probabilities that the first sample EEG signal is of each of various motor imagery types, respectively.
12. 12 . The apparatus according to claim 11 , wherein the attention mechanism comprises at least one of a spatial attention mechanism or a channel attention mechanism.
13. 13 . The apparatus according to claim 12 , wherein the first attention weighted network comprises a first spatial attention weighted network, a second convolutional layer, a first channel attention weighted network, and a third convolutional layer; and the attention feature obtaining code comprises: first spatial weighted sub-code configured to cause the at least one processor to perform, based on the first spatial attention weighted network, weighting processing based on the spatial attention mechanism on the first extracted feature map to obtain a first spatial feature map; second feature obtaining sub-code configured to cause the at least one processor to perform feature extraction on the first spatial feature map based on the second convolutional layer to obtain a second extracted feature map; first channel weighted sub-code configured to cause the at least one processor to perform, based on the first channel attention weighted network, weighting processing based on the channel attention mechanism on the second extracted feature map to obtain a first channel feature map; third feature obtaining sub-code configured to cause the at least one processor to perform feature extraction on the first channel feature map based on the third convolutional layer to obtain a third extracted feature map; and attention feature obtaining sub-code configured to obtain the attention feature map based on the first spatial feature map, the first channel feature map, and the third extracted feature map.
14. 14 . The apparatus according to claim 13 , wherein the first attention weighted network further comprises a second attention weighted network; and the attention feature obtaining sub-code further comprises: first fusion sub-code configured to cause the at least one processor to fuse the first spatial feature map, the first channel feature map, and the third extracted feature map to obtain a first fused feature map; and first attention weighted sub-code configured to cause the at least one processor to perform, through the second attention weighted network, weighting processing based on the attention mechanism on the first fused feature map to obtain the attention feature map.
15. 15 . The apparatus according to claim 14 , wherein the second attention weighted network comprises a second spatial attention weighted network and a second channel attention weighted network; and the first attention weighted sub-code comprises: first performing code configured to cause the at least one processor to perform, through the second channel attention weighted network, weighting processing based on the channel attention mechanism on the first fused feature map to obtain a second channel feature map; and second performing code configured to cause the at least one processor to perform, through the second spatial attention weighted network, weighting processing based on the spatial attention mechanism on the second channel feature map to obtain the attention feature map.
16. 16 . The apparatus according to claim 11 , wherein the first time-frequency feature obtaining code is configured to cause the at least one processor to perform continuous wavelet transform based on the at least two electrode signals to obtain the time-frequency feature maps of the at least two electrode signals.
17. 17 . A computer device, comprising a processor and a memory, the memory storing at least one computer instruction, and the at least one computer instruction being loaded and executed by the processor to implement the EEG signal classification method according to claim 1 .
18. 18 . A non-transitory computer-readable storage medium, storing at least one computer instruction, the at least one computer instruction being loaded and executed by a processor to implement the EEG signal classification method according to claim 1 .

Description

CROSS-REFERENCE TO RELATED APPLICATION(S) This application is a continuation application of International Application No. PCT/CN2022/077619, filed Feb. 24, 2022, which claims priority to Chinese Patent Application No. 202110220638.6, filed on Feb. 26, 2021, the disclosures of which are herein incorporated by reference in their entireties. FIELD The disclosure relates to the field of signal processing technologies, and in particular, to an electroencephalogram signal classification method and apparatus, a device, a storage medium, and a program product. BACKGROUND Electroencephalogram (EEG) records changes of electric waves during brain activity, and is the overall reflection of electrophysiological activities of brain nerve cells in the cerebral cortex or scalp surface. In the related art, a motor imagery-brain computer interface (MI-BCI) system has a wide application prospect in many fields, through which an external device can be controlled through electroencephalogram signals generated by imagining limb movements in the brain without any actual limb movements. Classification and recognition for motor imagery (MI) signals is a key operation in the MI-BCI system. SUMMARY Embodiments of the disclosure provide an electroencephalogram (EEG) signal classification method and apparatus, a device, a storage medium, and a program product. The technical solutions are as follows: According to an aspect of an example embodiment, an EEG signal classification method is provided, and performed by at least one processor. The method includes: obtaining a first EEG signal, the first EEG signal including at least two electrode signals, an electrode signal of the at least two electrode signals indicating an EEG signal generated by a target object in a spatial region corresponding to the electrode signal;obtaining time-frequency feature maps of the at least two electrode signals, a time-frequency feature map indicating a time-domain feature and a frequency-domain feature of the electrode signal;performing feature extraction based on the time-frequency feature maps of the at least two electrode signals to obtain a first extracted feature map, the first extracted feature map being fused with spatial features of the at least two electrode signals, and the spatial features of the at least two electrode signals being related to spatial regions corresponding to the at least two electrode signals;performing weighting processing based on an attention mechanism on the first extracted feature map to obtain an attention feature map of the first EEG signal; andobtaining a motor imagery type of the first EEG signal based on the attention feature map of the first EEG signal. According to an aspect of an example embodiment, an EEG signal classification method is provided, and performed by at least one processor. The method includes: obtaining a first sample EEG signal, the first sample EEG signal including at least two first sample electrode signals, and a first sample electrode signal indicating an EEG signal generated in a spatial region corresponding to the first sample electrode signal from a target object that performs motor imagery;obtaining first sample time-frequency feature maps of the at least two first sample electrode signals, a first sample time-frequency feature map indicating a time-domain feature and a frequency-domain feature of a corresponding first sample electrode signal;performing feature extraction on the first sample time-frequency feature maps of the at least two first sample electrode signals through a first convolutional layer in an EEG signal classification model, to obtain a first sample extracted feature map, the first sample extracted feature map being fused with spatial features of the at least two first sample electrode signals, and the spatial features of the at least two first sample electrode signals being related to spatial regions corresponding to the at least two first sample electrode signals;performing, based on an attention weighted network in the EEG signal classification model, weighting processing based on an attention mechanism on the first sample extracted feature map to obtain an attention feature map of the first sample EEG signal;obtaining a sample probability distribution of the first sample EEG signal based on the attention feature map of the first sample EEG signal, the sample probability distribution indicating probabilities that the first sample EEG signal is of each of a plurality of motor imagery types, respectively; andtraining the EEG signal classification model based on the sample probability distribution and a motor imagery type of the first sample EEG signal, the EEG signal classification model being configured to predict a motor imagery type of a first EEG signal. According to an aspect of an example embodiment, an EEG signal classification apparatus is provided. The apparatus includes: at least one memory configured to store program code; andat least one processor configured to