CN-116049406-B - Cross-domain emotion classification method based on contrast learning

CN116049406BCN 116049406 BCN116049406 BCN 116049406BCN-116049406-B

Abstract

The invention belongs to the field of big data analysis, and particularly relates to a cross-domain emotion classification method based on contrast learning. The method is used for enhancing feature extraction of emotion classification models on text information and analysis capability of cross-domain emotion, and comprises the following steps of S1, constructing a cross-domain emotion training model comprising a source domain feature extractor, a target domain feature extractor, an emotion classifier, a domain aligner and a contrast learner. S2, preprocessing the comment corpus to form a required comment data set. S3, designing an overall objective loss function required by model training, and S4, training a cross-domain emotion training model by using an Adam optimizer. S5, constructing a required cross-domain emotion classification network by utilizing a target domain feature extractor and an emotion classifier of the optimal cross-domain emotion training model. S6, performing cross-domain emotion classification on the target text information by using a cross-domain emotion classification network. The method solves the problem of insufficient emotion classification precision caused by the fact that the domain specific characteristics of the target domain are ignored in the existing method.

Inventors

HAN GUANGQING
CHEN JIE

Assignees

安徽大学

Dates

Publication Date: 20260508
Application Date: 20230206

Claims (10)

1. The cross-domain emotion classification method based on contrast learning is used for enhancing feature extraction of an emotion classification model on text information and analysis capability of cross-domain emotion, and is characterized by comprising the following steps of: S1, constructing a cross-domain emotion training model comprising a source domain feature extractor, a target domain feature extractor, an emotion classifier, a domain aligner and a contrast learner, wherein the data processing mode of the cross-domain emotion training model is as follows: (1) Extracting features of the source domain data Ds by adopting a source domain feature extractor to obtain source domain features Fs; (2) Performing data enhancement processing on the target domain data Dt to obtain enhanced data Dt'; (3) Extracting features of the target domain data Dt and the enhancement data Dt 'by using a target domain feature extractor sharing parameters with a source domain feature extractor to respectively obtain target domain features Ft and enhancement features Ft'; (4) The output of the source domain feature Fs is divided into two paths, one path is input into an emotion classifier, and the other path is input into a domain aligner, and the emotion classifier processes the output to obtain a corresponding emotion polarity classification result; (5) The target domain feature Ft extracted by the target domain feature extractor is divided into two paths, one path is input to the domain aligner, and the domain aligner calculates the difference of the two domains according to the source domain feature Fs and the target domain feature Ft; (6) The enhanced feature Ft 'extracted by the target domain feature extractor is input into a contrast learner, and the contrast learner combines the target domain feature Ft and the enhanced feature Ft' to extract specific features in the target domain; S2, acquiring a large number of comment corpus sets, and preprocessing the comment corpus sets to form a required comment data set, wherein the comment data set comprises source domain data Ds and target domain data Dt; s3, designing an overall objective loss function L required by training the cross-domain emotion training model, wherein the method comprises the following steps of: S31, using the cross entropy loss L sent as a source domain emotion classification loss of an emotion classifier according to a classification target; S32, setting an objective function for evaluating the performance of extracting specific characteristics of a target domain by a contrast learner as contrast loss L con according to the learning characteristics of the identification characteristics; s33, performing feature alignment on the source domain and target domain data to extract domain invariant features, and setting a loss function of a domain aligner as alignment difference loss L align ; S34, taking the minimized cross entropy loss L sent , the contrast loss L con and the alignment difference loss L align as the overall objective loss function L of the training phase of the cross-domain emotion training model, namely: L=L sent +αL con +βL align Wherein α and β are the weight factors of the contrast loss term and the alignment difference loss term, respectively; s4, training the cross-domain emotion training model by using an Adam optimizer, wherein the content of the training phase is as follows: s41, taking the data in the target domain data Dt and part of the source domain data Ds in the evaluation data set as a training set, and taking the data in the rest of the source domain data Ds as a verification set; S42, taking the designed overall target loss function L as an optimization function of a training stage; s43, setting various parameters of a training stage, including super parameters in a learning rate, iteration times, batch processing size and an overall target loss function L; S44, carrying out batch iterative training on the cross-domain emotion training model by adopting a training set, and storing model parameters after each round of training; s45, taking the accuracy rate as an evaluation index, and after each round of training is completed, verifying the trained network model through a verification set, and calculating a corresponding evaluation index; S5, selecting a cross-domain emotion training model with the best evaluation index after training, taking a target domain feature extractor as a feature extraction part, and taking an emotion classifier as a classification part to obtain a required cross-domain emotion classification network; And S6, sequentially carrying out feature extraction and analysis on the input target text information by utilizing the cross-domain emotion classification network, and further outputting a corresponding cross-domain emotion classification result.
2. The cross-domain emotion classification method based on contrast learning of claim 1, wherein in the cross-domain emotion training model constructed in step S1, the source domain feature extractor and the target domain feature extractor both use a pre-training language model BERT as a backbone for encoding context information of comment sentences, and both feature extractors are parameter-shared.
3. The cross-domain emotion classification method based on contrast learning as set forth in claim 2, wherein in the cross-domain emotion training model constructed in step S1, the emotion classifier is composed of a multi-layer perceptron MLP and a softmax layer, wherein the multi-layer perceptron comprises four layers, namely a full connection layer, a ReLU activation function layer, a droout layer and a full connection layer in sequence; The domain alignment consists of a multi-layer perceptron MLP and a CMD layer, the CMD layer consists of a plurality of fully connected layers, and the contrast learner consists of a multi-layer perceptron MLP and a contrastive leaning layer.
4. The cross-domain emotion classification method based on contrast learning of claim 1, wherein in step S1, a back-interpretation method is adopted to perform data enhancement on each comment text data in target domain data Dt, so as to obtain a positive sample pair of each comment text data.
5. The cross-domain emotion classification method based on contrast learning of claim 1, wherein in step S2, the preprocessing process of comment corpus is as follows: Firstly, loading a comment corpus and a pre-training language model, wherein the pre-training language model selects a BERT model or RoBERTa model, and then, carrying out text preprocessing and text data formatting on the comment corpus through the pre-training language model.
6. The method for cross-domain emotion classification based on contrast learning of claim 5, wherein the text pre-processing and text data formatting steps of the pre-trained language model are as follows: S001, using nltk word segmentation devices to segment the comment sentences, and separating the segmented token words by using spaces; s002, adding two special token words [ CLS ], [ SEP ] after the token sequence of the comment sentence segmentation, thus constructing a general input form of S= { [ CLS ], w 1 ,w 2 ...,w n , [ SEP ] }, Wherein n represents the total number of token words of the comment sentence, w n represents the n-th token of the comment sentence, [ CLS ] token is used for classification, and [ SEP ] token is used for separating two sentences; S003, carrying out filling processing on each comment sentence token word sequence to ensure that the lengths of the comment sentence token word sequences are 256, cutting sentences longer than 256, and filling zero for sentences less than 256; S004, using a word segmentation device tokenizer of the pre-training language model to carry out tokenize operation on each token word in the comment sentence, and obtaining source domain data Ds or target domain data Dt in the form of required batch data.
7. The cross-domain emotion classification method based on contrast learning of claim 1, wherein in step S31, the cross entropy loss L sent of emotion classifier is as follows: Wherein n s is the number of data samples in the original field, y i is the label of the ith sample of the data in the original field, and y i ' is the prediction label of the model on the ith sample of the data in the original field.
8. The cross-domain emotion classification method based on contrast learning of claim 1, wherein in step S32, contrast loss L con of contrast learner is as follows: Where k represents each term sequence number of the summation function, z i ,z j represents a pair of positive samples, sim (z i ,z j ) represents the cosine similarity of z i and z j , N is the size of a batch, t is a temperature superparameter which focuses the emphasis of model update to the negative case with difficulty and penalizes them accordingly, I is an equation indicating function, when k=i, the value of I is zero, otherwise the value is one.
9. The cross-domain emotion classification method based on contrast learning of claim 1, wherein in step S33, the alignment difference loss L align is a function as follows: wherein N represents the number of fully connected layers; And Respectively representing the characteristics of a source domain and a target domain passing through an ith full-connection layer; And Respectively representing the weights of full-connection layers of the source domain and the target domain of the ith layer; And Respectively representing the bias of the full-connection layers of the source domain and the target domain of the ith layer; F represents a ReLU activation function; the method is expressed as a K-order central moment calculation function and concretely comprises the following steps: Where A and B are two bounded random variables, the range interval is [ a, B ]; E (A) and E (B) are the expectations calculated for samples A and B, and C k (A) and C k (B) represent the center moment of the k-th order, i.e., the expected value that represents the k-th power of the off-center distance of the sample.
10. The cross-domain emotion classification method based on contrast learning of claim 1, wherein the learning rate is 2.0e-5, the temperature parameter of contrast loss L con in the contrast learner is 0.065, the batch size is 128, and the iteration round of training is 32 in each parameter of the training stage set in step S43.

Description

Cross-domain emotion classification method based on contrast learning Technical Field The invention belongs to the field of big data analysis, and particularly relates to a cross-domain emotion classification method based on contrast learning. Background With the continuous development of computer technology, virtual social networks are becoming an integral part of people's lives. In the information explosion age, comment texts on a network are more and more, and emotion classification technology plays an increasingly important role in automatically judging emotion polarities of texts. However, the text emotion classification technology requires a large number of label samples, and a large number of comment data which can be collected from a social network in the prior art are white data lacking in accurate emotion labels, which brings difficulty to the training of a network model related to cross-domain emotion classification. The source domain data is sufficient and has labels, and the target domain data is short of labels, which is common in the industry, so that part of researchers propose cross-domain emotion classification technology. The cross-domain emotion classification technology aims at improving emotion classification performance on a target domain through labeled source domain data and unlabeled target domain data. In the cross-domain emotion classification technology, domain offset between a source domain and a target domain is a main challenge in cross-domain emotion classification, and the domain offset problem is mainly a distribution difference problem between two domains. For example, words used in the book domain differ significantly from words used in the restaurant domain. Currently, unsupervised Domain Adaptation (UDA) techniques have been used to solve the domain offset problem. Essentially, the UDA utilizes unlabeled data in the target domain to minimize domain offset by characterizing Ji Yuanyu with the target domain. At present, related researches on cross-domain emotion classification can be mainly divided into two major categories, namely a task-independent method, wherein the methods comprise methods of divergence minimization, instance re-weighting, domain resistance training and the like, and some researches combine the task-independent method with a natural language processing-specific method or model. The method based on the divergence minimization is realized by minimizing the data distribution of the source domain and the target domain, so that the domain invariable representation can be obtained; based on example weighted cross-domain emotion classification, the distribution difference between the source domain and the target domain is reduced by giving a higher weight to a sample similar to the target domain in the training process, but the method inevitably generates a problem of negative migration in the migration process, and can reduce the emotion classification performance of the target domain; domain-based challenge training methods extract domain invariant features by minimizing the classification loss function of the source samples and the domain aliasing loss function of all samples. The second category is pivot-based methods that make up the gap in the domain by exploiting the correlation between pivots and non-pivots to learn domain invariant features. The most widely used pivot-based approach to date is Structural Correspondence Learning (SCL) and its variants. In SCLs, a pivot is generally defined as a word that often appears on the source domain and the target domain. The model can learn domain invariant features of axes efficiently, but is more challenging for non-axes because they have domain specific significance. Both of the above conventional schemes can well extract domain invariant features of the source domain and the target domain, but ignore specific features of the domain of the target domain. Therefore, the emotion classification accuracy of the network model is low, and the accuracy of the trained network model is often not expected under the condition of large data difference between the source domain and the target domain. Disclosure of Invention In order to solve the problem that the existing emotion classification technology only can extract domain invariant features of a source domain and a target domain and neglect domain specific features of the target domain, the invention provides a cross-domain emotion classification method based on contrast learning. The invention is realized by adopting the following technical scheme: A cross-domain emotion classification method based on contrast learning is used for enhancing feature extraction of an emotion classification model on text information and analysis capability of cross-domain emotion, and comprises the following steps: S1, constructing a cross-domain emotion training model comprising a source domain feature extractor, a target domain feature extractor, an emotion classifier, a domain