CN-121705849-B - Dual-contrast learning transducer cross-data set electroencephalogram emotion recognition method

CN121705849BCN 121705849 BCN121705849 BCN 121705849BCN-121705849-B

Abstract

The invention relates to a dual contrast learning transducer cross-data set electroencephalogram emotion recognition method, and belongs to the technical field of electroencephalogram emotion recognition and source domain independent domain self-adaptation. The method comprises the steps of obtaining a source domain electroencephalogram data set and a target domain electroencephalogram data set, constructing an electroencephalogram emotion recognition model comprising a lightweight time domain transducer feature extractor and a classifier, performing end-to-end supervised training on the electroencephalogram emotion recognition model by adopting the source domain electroencephalogram data set to obtain a pre-training model, inputting the target domain electroencephalogram data set into the frozen pre-training model to obtain initial target domain features and initial pseudo tags, optimizing the pre-training model through a loss function to obtain a trained electroencephalogram emotion recognition model, wherein the loss function comprises a first-stage loss, a second-stage loss and a third-stage loss, and inputting electroencephalogram data to be detected into the trained electroencephalogram emotion recognition model to obtain an electroencephalogram emotion recognition result. The invention can improve the accuracy of brain electricity emotion recognition.

Inventors

LI JINBAO
XIAO XIANGYU
GUO YAHONG
WEI NUO
GAO TIANLEI

Assignees

齐鲁工业大学(山东省科学院)
山东省人工智能研究院

Dates

Publication Date: 20260508
Application Date: 20260211

Claims (5)

1. The dual-contrast learning transducer cross-data set electroencephalogram emotion recognition method is characterized by comprising the following steps of: S1, acquiring a source domain brain electrical data set and a target domain brain electrical data set; S2, constructing an electroencephalogram emotion recognition model, wherein the electroencephalogram emotion recognition model comprises a lightweight time domain transducer feature extractor and a classifier, the lightweight time domain transducer feature extractor is formed by connecting a plurality of layers of time domain convolution and standard transducer encoder blocks in series, the classifier comprises a global average pooling layer and a full connection layer, the electroencephalogram emotion recognition model is subjected to end-to-end supervision training by adopting a source domain electroencephalogram data set to obtain a pre-training model, a target domain electroencephalogram data set is input into the frozen pre-training model, the lightweight time domain transducer feature extractor outputs initial target domain features of each target domain electroencephalogram data sample, the classifier outputs predictive probability distribution of each target domain electroencephalogram data sample, and the category corresponding to the maximum probability value is taken as an initial pseudo tag; S3, optimizing the pre-training model through a loss function to obtain a trained electroencephalogram emotion recognition model, wherein the loss function comprises a first-stage loss, a second-stage loss and a third-stage loss; The first-stage loss is information maximization loss, and the information maximization loss is expressed as follows: , Wherein, the Representing information maximization loss; Representing the average value of all samples in the target domain; representing target domain electroencephalogram data samples Is a predictive probability distribution of (1); representing target domain electroencephalogram data samples Belongs to the category of Conditional probability values of (2); representing an entropy function; representing a balance super parameter; representing an average predicted distribution of the batch; the second-stage loss adopts triple contrast learning loss and cross entropy classification loss based on high-confidence pseudo tag, wherein the triple contrast learning loss is obtained by example contrast loss Loss of category contrast Loss of contrast to cross-domain prototypes The weighted composition is formulated as follows: , Wherein, the Representing triple contrast learning loss; 、 And Representing three non-negative super-parameters, total loss in the second stage The formula of (c) is as follows: , Wherein, the Representing an average value for the high confidence target domain sample set; representing the balance weight coefficient; representing a cross entropy loss term; the third stage of loss adopts local consistency loss, and in the third stage, for each initial target domain feature, calculating Nearest neighbors of each Aiming at each initial target domain feature, adopting Euclidean distance as similarity measure, calculating K nearest neighbor sets in a feature space by a K nearest neighbor algorithm to obtain a reliable neighbor set, and losing local consistency Encouraging sample characteristics to be similar to those of its reliable neighbors, the formulation is as follows: , Wherein, the Representing initial target domain features A kind of electronic device Pseudo tag in neighbor Reliable neighbor sets that are identical and have a confidence above a dynamic adaptive threshold; Representing initial target domain features in the reliable neighbor set; S4, inputting the electroencephalogram data to be detected into a trained electroencephalogram emotion recognition model to obtain an electroencephalogram emotion recognition result.
2. The dual contrast learning transducer cross-dataset electroencephalogram emotion recognition method of claim 1, wherein the source domain electroencephalogram dataset is recorded as , wherein, Representation of the source domain electroencephalogram dataset Performing electroencephalogram test; Representation of the source domain electroencephalogram dataset True emotion type labels corresponding to individual electroencephalogram test times; is the total number of categories; Representing total brain electricity test times in a source domain brain electricity data set, wherein the target domain brain electricity data set is recorded as , wherein, Representation of target domain electroencephalogram dataset Performing electroencephalogram test; And representing the total number of electroencephalogram test times in the electroencephalogram data set of the target domain.
3. The method for recognizing brain electrical emotion of double contrast learning transducer across data set according to claim 2, wherein in the pre-training process, source domain prototypes of each category are calculated and stored by propagating all source domain brain electrical data samples forward The formula is as follows: , Wherein, the Indicating that the source domain electroencephalogram data set belongs to category Is a set of samples; indicating that the source domain electroencephalogram data set belongs to category Is a number of samples; The representation belongs to the category Is a source domain electroencephalogram data sample; representing a lightweight time domain transducer feature extractor; representative feature extractor Is provided for the learning of the parameter set.
4. The method for recognizing brain electrical emotion of double contrast learning transducer across data sets according to claim 3, wherein initial pseudo tags of brain electrical data samples of all target domains are used Purifying by adopting a dynamic threshold strategy, and taking the 75 th percentile value of the maximum value of all sample prediction probabilities as a dynamic self-adaptive threshold Reserving samples with confidence higher than a dynamic self-adaptive threshold to obtain a high-confidence sample set 。
5. The method for recognizing brain electrical emotion of double contrast learning transducer across data sets according to claim 4, wherein the instance contrast loss Encouraging different enhancement views of the same sample to be close together in feature space, formulated as follows: , Wherein, the And Representing two different features of the same initial target domain feature after random enhancement; Features representing all samples in the batch; Representing cosine similarity; representing a temperature parameter; Indicating the batch size; class contrast loss And (3) using the purified pseudo tag to zoom in the characteristics of the similar samples and push out the characteristics of the different samples, wherein the formula is as follows: , Wherein, the 、 A pseudo tag representing a jth sample, an ith sample, of the same class of samples; 、 Respectively representing the initial target domain characteristics of the ith sample and the initial target domain characteristics of the jth sample in the similar samples; Representing high confidence sample sets Initial target domain features for samples of different classes; Cross-domain prototype contrast loss Aligning the feature centers of each category in the target domain to pre-stored corresponding source domain prototypes, the formula being as follows: , Wherein, the Representing a prototype of the target domain, the pseudo tag belonging to the target domain Feature means for all samples of the class; representing the square of the euclidean distance.

Description

Dual-contrast learning transducer cross-data set electroencephalogram emotion recognition method Technical Field The invention belongs to the technical field of electroencephalogram emotion recognition and source domain independent domain self-adaptation, and particularly relates to a double contrast learning transform cross-data set electroencephalogram emotion recognition method. Background The brain electrical signal is used as an objective physiological index for reflecting brain nerve electrical activity, has outstanding value in the fields of emotion recognition, cognitive state monitoring, nerve disease auxiliary diagnosis and the like, but due to extremely strong individual specificity and scene dependence, the accuracy rate can be remarkably reduced when a high-performance model of source domain training is applied to a target domain, and a domain self-adaptive technology becomes a key of the brain electrolysis code trend practical application. The traditional model assumes that training is consistent with test data distribution, the early domain self-adaptive method needs to continuously access source domain original data, brain electrical data contains individual sensitive information, the brain electrical data is strictly constrained by privacy regulations, source domain data sharing is difficult to realize, and the limitation restricts the application of the brain electrical data. Under the deep learning background, although the domain self-adaptive method (such as CDTrans) based on the complex neural network such as a transducer improves the characteristic alignment effect, source domain data still needs to be read in real time in an reasoning or adaptation stage, so that the storage cost is high, the calculation delay is high and the privacy compliance risk exists during deployment. The current core challenge is to realize target domain self-adaptation on the premise of completely isolating source domain data and only utilizing a trained source domain model, construct a 'source domain irrelevant' paradigm, condense source domain knowledge and design a model self-evolution mechanism based on the target domain data so as to realize cross-domain robust generalization under privacy protection. Disclosure of Invention The invention aims to achieve the aim, and the aim is achieved by the following technical scheme: The invention provides a dual contrast learning transducer cross-data set electroencephalogram emotion recognition method, which comprises the following steps of: S1, acquiring a source domain brain electrical data set and a target domain brain electrical data set; S2, constructing an electroencephalogram emotion recognition model, wherein the electroencephalogram emotion recognition model comprises a lightweight time domain transducer feature extractor and a classifier, the lightweight time domain transducer feature extractor is formed by connecting a plurality of layers of time domain convolution and standard transducer encoder blocks in series, the classifier comprises a global average pooling layer and a full connection layer, the electroencephalogram emotion recognition model is subjected to end-to-end supervision training by adopting a source domain electroencephalogram data set to obtain a pre-training model, a target domain electroencephalogram data set is input into the frozen pre-training model, the lightweight time domain transducer feature extractor outputs initial target domain features of each target domain electroencephalogram data sample, the classifier outputs predictive probability distribution of each target domain electroencephalogram data sample, and the category corresponding to the maximum probability value is taken as an initial pseudo tag; S3, optimizing the pre-training model through a loss function to obtain a trained electroencephalogram emotion recognition model, wherein the loss function comprises a first-stage loss, a second-stage loss and a third-stage loss, the first-stage loss is information maximization loss, the second-stage loss adopts triple contrast learning loss and cross entropy classification loss based on high-confidence pseudo labels, and the third-stage loss adopts local consistency loss; S4, inputting the electroencephalogram data to be detected into a trained electroencephalogram emotion recognition model to obtain an electroencephalogram emotion recognition result. Further, the source domain electroencephalogram data set is recorded as, wherein,Representation of the source domain electroencephalogram datasetPerforming electroencephalogram test; Representation of the source domain electroencephalogram dataset True emotion type labels corresponding to individual electroencephalogram test times; is the total number of categories; Representing total brain electricity test times in a source domain brain electricity data set, wherein the target domain brain electricity data set is recorded as , wherein,Representation of target domain electroence