CN-121997275-A - Emotion recognition method for multi-band self-adaptive graph convolution

CN121997275ACN 121997275 ACN121997275 ACN 121997275ACN-121997275-A

Abstract

The invention discloses a multi-band self-adaptive graph convolution emotion recognition method, which introduces a self-adaptive topological graph which is self-adaptive according to a sample, and combines a memory attenuation mechanism to generate a time smooth topological graph, so that the self-adaptive topological graph is smoothed and constrained, and the influence of transient noise disturbance on topology estimation is reduced. Based on the joint constraint of the priori topological graph and the frequency band offset topological graph, a frequency band topological graph is constructed, and on the basis of keeping a certain frequency band sharing structure constraint, the description of frequency band difference connection is introduced, so that modeling complexity and uncertainty caused by respectively learning a complete dense topology for each frequency band are reduced. The fusion strategy for multi-band representation is provided, and by adaptively adjusting the information contributions of different frequency bands, the repeated effect of redundant frequency bands is restrained to a certain extent, and the information cooperation of complementary frequency bands is enhanced, so that more stable cross-frequency-band representation support is provided for emotion recognition.

Inventors

LI CHANGCHUN
YU ZHULIANG
ZENG XI
HOU RUIZHE
YU TIANYOU
GU ZHENGHUI

Assignees

华南理工大学

Dates

Publication Date: 20260508
Application Date: 20260327

Claims (4)

1. The emotion recognition method for the multi-band self-adaptive graph convolution is characterized by comprising the following steps of: S1, acquiring an original multichannel electroencephalogram signal, intercepting the original multichannel electroencephalogram signal according to a preset window length, intercepting a corresponding emotion recognition task each time to obtain a task electroencephalogram signal, segmenting the task electroencephalogram signal according to a preset time step to obtain multichannel short-time electroencephalogram signals of multiple time steps, and filtering and dividing the multichannel short-time electroencephalogram signals of all the time steps according to a preset frequency band by channel to obtain multi-frequency-band electroencephalogram signals; S2, sequentially taking out two-dimensional electroencephalogram characteristic tensors of channels-characteristics corresponding to each frequency band and each time step aiming at the four-dimensional electroencephalogram characteristic tensors, constructing a self-adaptive topological graph based on the two-dimensional electroencephalogram characteristic tensors and used for representing the transient functional connection between the frequency band and each channel under the time step, introducing a memory attenuation mechanism, namely, carrying out weighted fusion on the self-adaptive topological graph generated by different time steps to obtain a time-smooth topological graph so as to inhibit transient noise and keep time sequence consistency, constructing a prior topological graph shared by the frequency bands, learning a frequency band bias topological graph for describing the connection difference between the frequency bands for each frequency band, and fusing the time-smooth topological graph and the frequency band topological graph to obtain a frequency band time-varying topological graph of the corresponding frequency band at each time step, and stacking the frequency band time-varying topological graphs of each time step in time sequence, thus forming a frequency band time-varying topological graph sequence of the frequency band; S3, taking a three-dimensional electroencephalogram feature tensor of a time step-channel-feature corresponding to each frequency band of the four-dimensional electroencephalogram feature tensor as input, taking a frequency band time-varying topological graph sequence of the corresponding frequency band as constraint, extracting frequency band features fusing space-time information through time-space graph convolution to form frequency band feature vectors of each frequency band, constructing a frequency band similarity matrix, respectively constructing a redundancy suppression matrix for suppressing information redundancy and a complementary enhancement matrix for enhancing information complementation based on the frequency band similarity matrix, constructing a frequency band coupling matrix, combining the redundancy suppression matrix and the complementary enhancement matrix based on the frequency band coupling matrix to obtain frequency band weight vectors, carrying out weighted summation on the frequency band feature vectors of each frequency band based on the frequency band weight vectors to obtain a final fused electroencephalogram feature vector, inputting the fused electroencephalogram feature vectors into a classifier constructed by a full-connection layer, and outputting emotion category prediction results.
2. The emotion recognition method of multi-band adaptive graph convolution according to claim 1, wherein in step S1, a window length is set to be Second, the acquisition of task brain electrical signals is expressed as: ; in which the symbols are The expression "belonging to" indicates that the variable preceding the symbol belongs to a certain set following, Represents a set of real numbers, The number of channels is indicated and the number of channels is indicated, Represents the number of sampling points, if the sampling frequency is Then ; For task brain electric signal according to preset time step Segmenting in seconds to obtain the time step number of The multichannel short-time electroencephalogram signals of each time step are filtered and divided channel by channel according to a preset frequency band to obtain multichannel short-time electroencephalogram signals : ; In the formula, Is the number of the frequency bands, Represents the filtered electroencephalogram signal of the 1 st frequency band, Represents the filtered electroencephalogram signal of the 2 nd frequency band, Represent the first The filtered electroencephalogram signals of the individual frequency bands, Represent the first Filtering brain electrical signals of individual frequency bands; Subsequently, for the multi-band brain electrical signals Feature extraction is carried out, and four-dimensional electroencephalogram feature tensors taking frequency bands, time steps, channels and features as dimensions are constructed Wherein Representing the number of features.
3. The emotion recognition method of multi-band adaptive graph convolution according to claim 2, wherein said step S2 comprises the steps of: S21, aiming at four-dimensional electroencephalogram characteristic tensors Taking out each frequency band in turn And time step Corresponding channel-feature two-dimensional brain electrical feature tensor And respectively pass through And Obtaining a query matrix And key matrix Wherein And As a learnable linear mapping weight matrix, Represents the operation of the transposition, Representation of And (3) with Finally generating self-adaptive topological graph To characterize the transient functional connection between the channels of the brain network in the frequency band and the time step: ; in the formula, Representing normalized mapping, symbols Representing the multiplication by element, A mask matrix constructed based on the ROI or the geometric rule is used for shielding node pairs which do not accord with the preset connection rule; S22, introducing a memory attenuation mechanism, and obtaining a time smooth topological graph based on the self-adaptive topological graph I.e. for frequency bands In the time step In the time-course of which the first and second contact surfaces, In the time step When the current time is stepped Is an adaptive topology of (a) And last time step Is a time-smoothed topology of (2) Weighting and fusing to obtain the current time step Is a time-smoothed topology of (2) Thereby suppressing transient noise and maintaining timing consistency: ; in the formula, Is of frequency band The memory coefficient of which takes on a value depending on the frequency band; s23, taking an electroencephalogram channel as a graph node, comprehensively considering brain region ROI division, channel geometric proximity relation and left-right hemispherical symmetry relation, and constructing a priori topological graph shared by cross frequency bands Wherein the connection weight of the channels in the same ROI is higher than the connection weight of the channels in different ROIs, on the basis, the connection difference of different frequency bands is described, and each frequency band is Setting frequency band offset topological graph And is opposite to Applying sparse constraint to learn to inhibit redundant connection and promote topology interpretability, and finally fusing the two to obtain frequency band Frequency band topology of (a) : ; S24, smoothing the time smooth topological graph of each frequency band Topological graph of frequency band Fusion to obtain frequency band Time step Frequency band time-varying topology map of (2) : ; Frequency band time-varying topological graph of each time step Stacking in time sequence to form a time-varying topological graph sequence of the frequency band : ; In the formula, Representing frequency bands A frequency band time-varying topological graph of the 1 st time step, Representing frequency bands A frequency band time-varying topological graph of the 2 nd time step, Representing a representation frequency band First, the A frequency band time-varying topology of individual time steps, Representing frequency bands First, the A time-step frequency band time-varying topology graph.
4. A method of emotion recognition by multi-band adaptive graph convolution according to claim 3, wherein said step S3 comprises the steps of: S31, three-dimensional brain electrical characteristic tensor of time step-channel-characteristic corresponding to each frequency band of four-dimensional brain electrical characteristic tensor As input, a frequency band time-varying topological graph sequence of a corresponding frequency band For constraint, a space-time joint convolution is performed by a space-time diagram convolution model STGCN and along And Global pooling is carried out to generate frequency band feature vectors , wherein, The dimension of the frequency band feature vector; S32, will Feature vectors of the frequency bands are stacked to obtain a matrix : ; In the formula, A frequency band feature vector representing the 1 st frequency band, A frequency band feature vector representing the 2 nd frequency band, Represent the first The frequency band characteristic vector of each frequency band, Represent the first Frequency band characteristic vectors of the individual frequency bands; For a pair of Normalizing according to the rows to obtain a matrix Constructing a frequency band similarity matrix Wherein Column vectors representing all 1's of elements, then constructing redundancy suppression matrix based on the frequency band similarity matrix Complementary enhancement matrix , wherein, As a function of the Sigmoid, To suppress intensity coefficients; S33, constructing a frequency band coupling matrix And incorporate redundancy inhibition matrix Complementary enhancement matrix Obtaining a frequency band weight vector : ; In the formula, In order to cooperate with the enhancement factor, The weight coefficient representing the 1 st frequency band, The weight coefficient representing the 2 nd band, Represent the first The weight coefficient of each frequency band, Represent the first Weight coefficients of the individual frequency bands; s34, weighting and summing the characteristic vectors of each frequency band to obtain a final fused electroencephalogram characteristic vector : ; The brain electrical feature vector will then be fused Input classifier constructed by full connection layer, output emotion classification predictive probability distribution vector : ; In the formula, And finally, the category corresponding to the maximum probability is used as an emotion recognition result.

Description

Emotion recognition method for multi-band self-adaptive graph convolution Technical Field The invention relates to the technical field of electroencephalogram signal decoding, in particular to a multi-band self-adaptive graph convolution emotion recognition method. Background The brain electrical signal has the characteristics of high time resolution and non-invasive acquisition, and has application value in the fields of emotion recognition, brain-computer interface, nerve state evaluation and the like. However, the signal-to-noise ratio of the electroencephalogram signal is low, obvious individual differences and distribution shifts exist among different tested, and the problems of reduced generalization performance and insufficient stability easily occur in a task of identifying emotion across tested. In the prior art, a convolutional neural network or a cyclic neural network is often adopted to model multi-channel signals or frequency domain characteristics thereof, but the modeling of non-European topology dependency relationship among channels is insufficient, and the brain region functional connection mode is difficult to be described. In recent years, the graph convolution neural network characterizes the functional connection relation among channels by constructing an adjacent matrix, and provides a new modeling method for brain electric emotion recognition. However, the adjacency matrix of the prior graph method is statically defined according to the prior of electrode space distance or fixed coherence and the like, and lacks self-adaptive learning capability for dynamic function connection under task driving, and meanwhile, the connection structure possibly contains more redundant side weights, and noise propagation can be introduced and overfitting risks are increased. In addition, the multi-band information is usually simply spliced or independently weighted and fused, so that the contribution of the redundant band and the complementary band is difficult to effectively distinguish. Disclosure of Invention The invention aims to provide a multi-band self-adaptive graph convolution emotion recognition method, which is used for realizing joint modeling of an electroencephalogram space topological structure and time dynamic change by constructing a frequency band time-varying topological graph, updating the dynamic graph topology by a memory attenuation mechanism, introducing a frequency band fusion strategy to inhibit redundant frequency bands and enhance complementary frequency band cooperative contribution, so that more effective modeling and fusion of the electroencephalogram space-time topology are realized. In order to achieve the purpose, the technical scheme provided by the invention is that the emotion recognition method for the multi-band self-adaptive graph convolution comprises the following steps: S1, acquiring an original multichannel electroencephalogram signal, intercepting the original multichannel electroencephalogram signal according to a preset window length, intercepting a corresponding emotion recognition task each time to obtain a task electroencephalogram signal, segmenting the task electroencephalogram signal according to a preset time step to obtain multichannel short-time electroencephalogram signals of multiple time steps, and filtering and dividing the multichannel short-time electroencephalogram signals of all the time steps according to a preset frequency band by channel to obtain multi-frequency-band electroencephalogram signals; S2, sequentially taking out two-dimensional electroencephalogram characteristic tensors of channels-characteristics corresponding to each frequency band and each time step aiming at the four-dimensional electroencephalogram characteristic tensors, constructing a self-adaptive topological graph based on the two-dimensional electroencephalogram characteristic tensors and used for representing the transient functional connection between the frequency band and each channel under the time step, introducing a memory attenuation mechanism, namely, carrying out weighted fusion on the self-adaptive topological graph generated by different time steps to obtain a time-smooth topological graph so as to inhibit transient noise and keep time sequence consistency, constructing a prior topological graph shared by the frequency bands, learning a frequency band bias topological graph for describing the connection difference between the frequency bands for each frequency band, and fusing the time-smooth topological graph and the frequency band topological graph to obtain a frequency band time-varying topological graph of the corresponding frequency band at each time step, and stacking the frequency band time-varying topological graphs of each time step in time sequence, thus forming a frequency band time-varying topological graph sequence of the frequency band; S3, taking a three-dimensional electroencephalogram feature tensor of a time step-channel-feature cor