CN-121971107-A - Emotion recognition method, system, equipment and medium based on electroencephalogram signals

CN121971107ACN 121971107 ACN121971107 ACN 121971107ACN-121971107-A

Abstract

The invention provides an emotion recognition method, system, equipment and medium based on an electroencephalogram signal, belonging to the technical field of brain-computer interfaces and emotion calculation, wherein the method comprises the following steps: the brain region flow which has both physiological rationality and data adaptability is divided by adopting a double-layer strategy of priori macroscopic brain region flow and self-supervision self-adaptation sub-flow division. Then, a hierarchical element learning mechanism is introduced, and the local detail and the global trend difference of the individual are respectively adapted through a local element learner and a global element learner. And finally, constructing a local and global feature alignment and cross-brain region fusion module, and realizing semantic consistency and complementarity mining of local and global features. The method solves the problem of low EEG emotion recognition accuracy under the cross-test scene.

Inventors

LIU HAO
WANG XIULE
Cui Jiateng
DONG YINDONG
Meng Ruxin
YE YONG
YUE ZHENYU
WU GUODONG
LU QIANG
ZHOU XINJUN
XING YUBO

Assignees

安徽农业大学

Dates

Publication Date: 20260505
Application Date: 20260407

Claims (10)

1. An emotion recognition method based on an electroencephalogram signal is characterized by comprising the following steps of: acquiring an electroencephalogram signal, and extracting time-frequency characteristics of a plurality of channels in the electroencephalogram signal; dividing the channels into predefined macroscopic brain region flows, and dividing the sub-flows in each macroscopic brain region flow based on the time-frequency characteristics of each channel contained in the macroscopic brain region flows to obtain a plurality of brain region functional sub-flows; The method comprises the steps of respectively extracting local characteristics of a sub-stream and global characteristics of the sub-stream after individual difference adaptation for each brain region functional sub-stream, dynamically adjusting parameters of a local characteristic extraction network according to individual local characteristic statistics through a local element learner to adapt to individual local differences, dynamically adjusting parameters of a global characteristic extraction network according to individual global characteristic statistics through a global element learner to adapt to individual global differences, inputting the time-frequency characteristics corresponding to each channel contained in the sub-stream into the local characteristic extraction network and the global characteristic extraction network after parameter adaptation, and respectively extracting the local characteristics of the sub-stream and the global characteristics of the sub-stream; Fusing the local characteristics of the substream with the global characteristics of the substream to obtain complete characteristics of substreams of the brain region stream; The method comprises the steps of carrying out interactive fusion on complete characteristics of all sub-streams belonging to the same macroscopic brain region stream to obtain total characteristics of the macroscopic brain region stream, carrying out cross-brain region fusion on the total characteristics of all macroscopic brain region streams to obtain global integrated characteristics, and outputting emotion recognition results based on the global integrated characteristics.
2. The method according to claim 1, wherein the network extracted sub-stream local features based on the parameter adapted local features comprises the steps of: The method comprises the steps of inputting time-frequency characteristics into a pre-trained mixed expert model, extracting local characteristics from different time scales and frequency band interaction angles through a plurality of expert networks, extracting local space-time statistical information of the time-frequency characteristics through a gating network, wherein the local space-time statistical information comprises a mean value, a variance and a correlation coefficient between frequency bands of each time window; Inputting the multi-scale local preliminary features into a bilateral feature interaction and enhancement module, mapping the multi-scale local preliminary features into a query vector Q, a key vector K and a value vector V through linear projection, and adding time position codes into the Q and K to reserve the sequential information of each time window; And adding the multi-scale local preliminary features and the local interaction features through residual connection and layer normalization operation, and performing normalization processing to obtain the finally-adapted sub-stream local features.
3. The method according to claim 2, wherein the parameters of the local feature extraction network are dynamically adjusted by the local meta-learner according to the local feature statistics of the individual to adapt to the individual local differences, comprising the steps of: inputting the local characteristic statistics of the current tested individuals into a pre-trained local element learner, wherein the local element learner consists of a multi-layer fully-connected network, and outputs the local element learner as parameter adjustment quantity aiming at the current local characteristic extraction network Wherein Corresponding to the parameter adjustment amount of the mixed expert model, The parameter adjustment quantity of the corresponding bilateral feature interaction and enhancement module, wherein the individual local feature statistic is the DE average value of the tested beta frequency band; According to the parameter adjustment quantity Initial population training parameters for the hybrid expert model and bilateral feature interaction and enhancement module Performing individual update to obtain the adapted network parameters : Wherein Is a preset adaptation coefficient and is used for controlling the amplitude of the individual adjustment.
4. The method according to claim 1, wherein extracting the network extracted sub-stream global features based on the parameter adapted global features, comprises the steps of: carrying out serialization processing on the time-frequency characteristics to obtain time sequence characteristics; Inputting the time sequence characteristics into a pre-trained state space model, wherein the state space model adopts a Mamba structure and comprises a plurality of layers of selective state space blocks, each layer of selective state space blocks performs state space projection on the input time sequence characteristics through a selective state space mechanism and dynamically selects information to be reserved or forgotten based on the input characteristics; The method comprises the steps of inputting the time sequence characteristics and the intermediate hidden state of a state space model into a context time sequence interactive relay module, wherein the relay module comprises a circulating memory unit and a gating fusion mechanism, the circulating memory unit is used for maintaining the memory state of a history time window, updating the memory state at the current moment through the gating fusion mechanism, traversing all time windows, and fusing the final memory state as a global characteristic after context enhancement with an initial global characteristic to obtain a sub-stream global characteristic.
5. The method according to claim 4, wherein the step of dynamically adjusting parameters of the global feature extraction network to adapt to the global differences of the individuals by the global meta learner according to global feature statistics of the individuals comprises the steps of: Inputting the individual global feature statistics to be tested into a pre-trained global element learner, wherein the global element learner is composed of a plurality of layers of fully connected networks and outputs the parameters as parameter adjustment quantity aiming at the current global feature extraction network Wherein Corresponding to the parameter adjustment amount of the state space model, The global characteristic statistic is a sliding average value of the DE characteristics of 5 frequency bands; According to the parameter adjustment quantity And carrying out personalized updating on the initial group training parameters of the state space model and the context time sequence interaction relay module to obtain the adapted network parameters.
6. The method of claim 1, wherein the fusing the local sub-stream features with the global sub-stream features to obtain the complete sub-stream features of the brain region stream specifically comprises splicing and fusing the local sub-stream features with the global sub-stream features, mapping the local sub-stream features and the global sub-stream features to a same semantic space, introducing self-supervision alignment loss to maximize similarity, and forcing the local sub-stream features and the global sub-stream features to be semantically consistent to output the complete sub-stream features of the brain region stream.
7. The method according to claim 1, wherein the sub-stream division is performed inside each of the macro-brain region streams based on the time-frequency characteristics of each channel contained therein, so as to obtain a plurality of brain region functional sub-streams, and the method comprises the following steps: Generating an embedded representation for time-frequency characteristics of each channel within the current macroscopic brain region stream; Based on self-supervision contrast learning, constructing a positive sample pair and a negative sample pair, wherein embedded representations of two channels are randomly sampled from the same macroscopic brain region flow as the positive sample pair, and embedded representations of one channel are randomly sampled from different macroscopic brain region flows as the negative sample pair; Calculating a contrast loss function based on the positive and negative pairs of samples, the contrast loss function being defined as the negative logarithm of the ratio of the positive pair of samples similarity to the sum of all pairs of samples similarity; Minimizing the contrast loss function through a back propagation algorithm to pull the distance of the positive sample pair embedded representation in the feature space, and meanwhile, pushing the distance of the negative sample pair embedded representation in the feature space to obtain a channel embedded representation after contrast learning; and dynamically determining the number of sub-streams contained in each macroscopic brain region stream and the attribution of the channels through a differential clustering algorithm based on the channel embedded representation after the comparison and learning, so as to form the functional sub-streams of the brain regions.
8. An electroencephalogram signal-based emotion recognition system, comprising: The extraction module is used for acquiring electroencephalogram signals and extracting time-frequency characteristics of a plurality of channels in the electroencephalogram signals; the system comprises a dividing module, a processing module and a processing module, wherein the dividing module is used for dividing the channels into predefined macroscopic brain region flows, and performing sub-flow division in each macroscopic brain region flow based on the time-frequency characteristics of each channel contained in the macroscopic brain region flow to obtain a plurality of brain region functional sub-flows; The characteristic modeling module is used for respectively extracting the local characteristic of the sub-stream and the global characteristic of the sub-stream after the adaptation of the individual difference for each brain region functional sub-stream, wherein the parameters of the local characteristic extraction network are dynamically adjusted by the local element learner according to the local characteristic statistics of the individual to adapt to the individual local difference, and the parameters of the global characteristic extraction network are dynamically adjusted by the global element learner according to the global characteristic statistics of the individual to adapt to the individual global difference; The fusion module is used for fusing the local characteristics of the substream with the global characteristics of the substream to obtain the complete characteristics of the substream of the brain region stream; The identification module is used for carrying out interactive fusion on complete characteristics of all sub-streams belonging to the same macroscopic brain region stream to obtain total characteristics of the macroscopic brain region stream, carrying out cross-brain region fusion on the total characteristics of all macroscopic brain region streams to obtain global integrated characteristics, and outputting emotion identification results based on the global integrated characteristics.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of the preceding claims 1 to 7 when the program is executed.
10. A readable storage medium storing a computer program which, when executed by a processor, implements the method of any one of claims 1-7.

Description

Emotion recognition method, system, equipment and medium based on electroencephalogram signals Technical Field The invention belongs to the technical field of brain-computer interfaces and emotion calculation, and particularly relates to an emotion recognition method, system, equipment and medium based on electroencephalogram signals. Background Emotion is used as a core driving force for human cognition and behaviors, and the accurate identification of emotion has important application value in the fields of human-computer interaction, mental health monitoring, intelligent education and the like. Electroencephalogram is an ideal physiological signal source for emotion recognition because of noninvasive property, high time resolution and characteristics of directly reflecting brain nerve activity, and emotion recognition technology based on EEG has become a research hotspot in the field of emotion calculation. In recent years, deep learning methods have made significant progress in EEG emotion recognition by virtue of strong feature learning capabilities. Convolutional Neural Network (CNN), cyclic neural network (RNN), and transducer models achieve efficient classification of emotional states, such as positive, neutral, and negative, by extracting spatiotemporal features of EEG signals. However, the existing method has three key challenges when facing the complexity and the specificity of the brain signal (1) the dividing strategy of brain region functional flow has limitation. The traditional method is difficult to adapt to the distribution difference of brain data under different tested and different tasks due to the fact that brain area flows are partitioned according to fixed physiological priori, or is easy to break away from physiological relevance of brain functions due to the fact that pure data driving self-adaptive partitioning is adopted, so that feature physical meaning is fuzzy, the interpretability and robustness of a model are affected, and (2) the individual difference adaptation capability is insufficient. EEG signals are obviously affected by individual factors such as age, sex, physiological state and the like, and the distribution of different tested brain electrical characteristics has obvious heterogeneity. The existing model is mostly based on group data training, lacks a dynamic adaptation mechanism aiming at individual details, greatly reduces generalization performance under a new tested or small sample scene, and has the advantages of partial short-time characteristics and full-time trend cracking. The emotional experience is the result of local neural activity combined with the dynamic behavior of the overall situation length. The existing method is often used for independently modeling local or global features, ignoring the inherent association of the local or global features, causing incomplete feature characterization and limiting further improvement of recognition accuracy. In summary, in the prior art, problems of stiff brain region division strategy, weak individual difference adaptation capability and local and global feature splitting exist, so that EEG emotion recognition accuracy is low in a cross-test scene. Disclosure of Invention In order to solve the problem of low EEG emotion recognition accuracy under a cross-test scene, the invention provides an emotion recognition method, system, equipment and medium based on an electroencephalogram signal. According to a first aspect of the embodiment of the invention, there is provided an emotion recognition method based on an electroencephalogram signal, comprising the following steps: acquiring an electroencephalogram signal, and extracting time-frequency characteristics of a plurality of channels in the electroencephalogram signal; dividing the channels into predefined macroscopic brain region flows, and dividing the sub-flows in each macroscopic brain region flow based on the time-frequency characteristics of each channel contained in the macroscopic brain region flows to obtain a plurality of brain region functional sub-flows; The method comprises the steps of respectively extracting local characteristics of a sub-stream and global characteristics of the sub-stream after individual difference adaptation for each brain region functional sub-stream, dynamically adjusting parameters of a local characteristic extraction network according to individual local characteristic statistics through a local element learner to adapt to individual local differences, dynamically adjusting parameters of a global characteristic extraction network according to individual global characteristic statistics through a global element learner to adapt to individual global differences, inputting the time-frequency characteristics corresponding to each channel contained in the sub-stream into the local characteristic extraction network and the global characteristic extraction network after parameter adaptation, and respectively extracting the local