CN-122019775-A - Text attribute map node classification method based on hybrid variation expectation maximization

CN122019775ACN 122019775 ACN122019775 ACN 122019775ACN-122019775-A

Abstract

The invention provides a text attribute map node classification method based on hybrid variation expectation maximization, and belongs to the technical field of data mining. The method comprises the steps of constructing a basic double-module framework based on a pre-training language model and a graphic neural network, initializing, performing M-step forward propagation once to generate a structural pseudo tag, performing E-step training on supervision updating of the graphic neural network by the pre-training language model, performing M-step training on supervision updating of the pre-training language model by the graphic neural network, performing alternate iteration on the E-step training and the M-step training to obtain a trained language model and the graphic neural network, and predicting by a test node by using the trained language model and the graphic neural network to obtain a classification result. The invention effectively solves the problem of mutual cutting between embedding alignment and label supervision through joint optimization of embedding and labels, so that the learned node representation has structure-semantic consistency and task discrimination capability at the same time, and is obviously superior to the existing method on a plurality of text attribute graph reference data sets.

Inventors

LIANG SHUANG
QIN KE
LUO GUANGCHUN
MA YIZHUO
QING YU
Cai Wudong
CHEN QIZHI

Assignees

电子科技大学

Dates

Publication Date: 20260512
Application Date: 20260415

Claims (9)

1. A text attribute map node classification method based on hybrid variation expectation maximization, the method comprising the steps of: step S1, constructing a basic double-module framework based on a pre-training language model and a graphic neural network and initializing; S2, performing M-step forward propagation once on an adjacent matrix and initial semantic embedding among nodes to generate a structural pseudo tag; S3, performing E-step training on the pre-training language model under the supervision and updating of the graphic neural network; step S4, the graphic neural network executes M-step training under the supervision and update of the pre-training language model; step S5, alternately iterating E-step training and M-step training, continuously optimizing the pre-training language model and the graphic neural network under mutual supervision, and finally obtaining the trained language model and the trained graphic neural network; and S6, predicting by using the trained language model and the graph neural network by the test node to obtain a classification result.
2. The text attribute map node classification method based on hybrid variational desire maximization of claim 1, wherein step S1 comprises the steps of: s11, selecting a pre-training language model and a graphic neural network as a basic double-module framework, and initializing the pre-training language model and the graphic neural network; step S12, acquiring input data from a data set and setting control parameters; And S13, generating initial semantic embedding.
3. The text attribute map node classification method based on hybrid variational desire maximization according to claim 2, wherein step S2 comprises the steps of: S21, structure embedding and extraction, namely, the graph neural network takes an adjacent matrix among nodes and initial semantic embedding as input, and the neighborhood structure information is accumulated through a graph volume to obtain the structure embedding of the nodes; Step S22, mapping distribution parameters, namely mapping the structural embedding of each node into structural embedding mean value and standard deviation through a second MLP linear layer, so that the structural embedding obeys Gaussian distribution; s23, gaussian sampling, namely sampling Gaussian distribution obeyed by structural embedding by utilizing a re-parameterization skill to obtain M-step sampling distribution examples; And S24, generating a structure pseudo tag, namely inputting all sampling distribution examples into another graph neural network layer, and generating the structure pseudo tag of M steps through graph convolution and classification mapping.
4. A text attribute map node classification method based on hybrid variational desire maximization according to claim 3, wherein step S3 comprises the steps of: s31, inputting a node text sequence into a pre-training language model to obtain node text semantic embedding; step S32, mapping the text semantic embedding of each node into a semantic embedding mean value and a standard deviation through a first MLP linear layer, so that the semantic embedding obeys Gaussian distribution; S33, constructing MSE loss in the step E, and updating a pre-training language model by back propagation; step S34, acquiring an E-step sampling distribution example through Gaussian sampling based on Gaussian distribution described by semantic embedding mean and standard deviation; S35, all sampling distribution examples generate semantic pseudo tags in the step E through a second linear layer; step S36, constructing a cross entropy loss function of the step E, and updating parameters of a second linear layer as a target; And step S37, embedding the mean value, the standard deviation and the semantic pseudo tag into the semantics generated in the step E, and playing a role in supervision in the next step M of training.
5. The text attribute map node classification method based on hybrid variational desire maximization of claim 4 wherein step S4 comprises the steps of: s41, structure embedding and extraction, namely, the graph neural network takes an adjacent matrix among nodes and node semantic embedding as input, and the neighborhood structure information is accumulated through a graph volume to obtain the structure embedding of the nodes; Step S42, mapping distribution parameters, namely mapping the structural embedding of each node into structural embedding mean values and standard deviations through a second MLP linear layer, so that the structural embedding obeys Gaussian distribution; S43, constructing MSE loss updating in the M steps; s44, gaussian sampling, namely sampling Gaussian distribution obeyed by structural embedding by utilizing a re-parameterization skill to obtain an M-step sampling distribution example; S45, generating a structure pseudo tag, namely inputting all sampling distribution examples into another graph neural network layer, and generating the structure pseudo tag in the step M through graph convolution and classification mapping; Step S46, constructing cross entropy loss of the M steps, and updating parameters of the graph neural network for the target; step S47, embedding the structure generated in the M steps into the mean value Standard deviation of And structured pseudo tag And plays a role in supervision in the next step E training.
6. The method for classifying nodes of a text attribute map based on hybrid variational expectation maximization of claim 4, wherein the MSE loss at step E is expressed as follows: Wherein, the Representing MSE loss in step E; Representing an embedding dimension; Representing the number of nodes; And Respectively represent the first Semantic embedding means and standard deviations of the individual nodes; 、 Respectively represent the first The structure of the individual nodes embeds the mean and standard deviation.
7. The method for classifying nodes of a text attribute map based on hybrid variational expectation maximization of claim 5, wherein the MSE loss at step M is expressed as follows: Wherein, the Representing MSE loss in M steps; Representing an embedding dimension; Representing the number of nodes; And Respectively represent the first Semantic embedding means and standard deviations of the individual nodes; 、 Respectively represent the first The structure of the individual nodes embeds the mean and standard deviation.
8. The method for classifying text attribute map nodes based on hybrid variational expectation maximization of claim 4 wherein the cross entropy loss function of step E is expressed as follows: Wherein, the Representing the cross entropy loss of step E; Representing a set of unknown tag nodes; representing a set of known label nodes; Representing the distribution of the structural pseudo labels output in the M-step initialization or iteration process, keeping unchanged in the E-step optimization process, and playing a role in supervision; Representation of E step sampling distribution examples of the individual nodes; Represent the first Category labels of individual nodes; representing a set of known tags that are to be used, , For a set of subscripts of a labeled node, Is a node set; A set of labels representing unlabeled nodes, A subscript set of unlabeled nodes; representation divide by Label sets of other unlabeled nodes except the individual nodes; Representing a text attribute map; representing semantic tag distribution learned by the second linear layer in the step E; Representing the weight coefficient.
9. The method for classifying text attribute map nodes based on hybrid variational expectation maximization of claim 5, wherein the cross entropy loss of M steps is expressed as follows: Wherein, the Representing the cross entropy loss of the M steps; Representing the weight coefficient; Represent the first M steps of sampling distribution examples of the individual nodes; the semantic pseudo label distribution output in the step E is represented and kept unchanged in the optimization process in the step M, so that a supervision effect is achieved; representing the structural label learned by the M-step graph neural network; Represent the first Category labels of individual nodes; representing a set of known tags that are to be used, , For a set of subscripts of a labeled node, Is a node set; A set of labels representing unlabeled nodes, A subscript set of unlabeled nodes; representation divide by Label sets of other unlabeled nodes except the individual nodes; representing a text attribute map.

Description

Text attribute map node classification method based on hybrid variation expectation maximization Technical Field The invention belongs to the technical field of data mining, and particularly relates to a text attribute map node classification method based on hybrid variation expectation maximization. Background The text attribute map (Text Attribute Graphs) is a multidimensional representation method integrating linguistic features and topological relations, and the core aim is to reveal semantic relevance and attribute distribution implicit in text data through structural modeling. In a text attribute graph, nodes (nodes) are represented by discrete text units (e.g., words, sentences, paragraphs) that are in conventional natural language processing, and edges (edges) are constructed from grammatical dependencies, co-occurrence frequencies, or semantic similarities between these text units. Text attribute maps are widely found in many real-world scenarios, such as citation networks and social networks. In these scenarios, it is often necessary to predict labels of unlabeled nodes from labeled node labels, i.e., supervised node classification problems. Achieving accurate label prediction on text property graphs relies on building an embedding that has both structure-semantic consistency and task-specific discriminative power. In contrast, effective embedding alignment relies on task-specific signals to emphasize relevant features. Methods focused on embedding alignment typically ignore downstream task goals, resulting in the embedding lacking the discrimination capability required for accurate label prediction. On the contrary, the task driven approach, which only focuses on label supervision, ignores the importance of integrating structure and semantic information, resulting in a model that is easily over-fitted. This separation ultimately hampers the robustness and uniformity of node characterization, limiting their performance on downstream tasks. A natural solution to address the cyclic dependencies between embedded alignment and tag supervision is to optimize these goals simultaneously. However, joint optimization complicates the loss landscape, making stable and efficient convergence more difficult. In particular, the embedding alignment aims at maximizing mutual information between text and structural embeddings, ensuring robustness to node feature and graph topology changes. In contrast, label supervision focuses on minimizing the prediction error of downstream tasks, which requires embedding to be highly separable along task-specific decision boundaries. While these goals are not inherently contradictory, their joint optimization generally results in higher variance and slower convergence speed. Disclosure of Invention The invention aims to provide a text attribute graph node classification method based on hybrid variation expectation maximization, which aims to solve the technical problem of embedded-tag cyclic dependence in the prior art and improve the node classification capability of a model. In order to solve the technical problems, the specific technical scheme of the invention is as follows: A text attribute map node classification method based on hybrid variational expectation maximization, the method comprising the steps of: step S1, constructing a basic double-module framework based on a pre-training language model and a graphic neural network and initializing; S2, performing M-step forward propagation once on an adjacent matrix and initial semantic embedding among nodes to generate a structural pseudo tag; S3, performing E-step training on the pre-training language model under the supervision and updating of the graphic neural network; step S4, the graphic neural network executes M-step training under the supervision and update of the pre-training language model; step S5, alternately iterating E-step training and M-step training, continuously optimizing the pre-training language model and the graphic neural network under mutual supervision, and finally obtaining the trained language model and the trained graphic neural network; and S6, predicting by using the trained language model and the graph neural network by the test node to obtain a classification result. Further, step S1 includes the steps of: s11, selecting a pre-training language model and a graphic neural network as a basic double-module framework, and initializing the pre-training language model and the graphic neural network; step S12, acquiring input data from a data set and setting control parameters; And S13, generating initial semantic embedding. Further, step S2 includes the steps of: S21, structure embedding and extraction, namely, the graph neural network takes an adjacent matrix among nodes and initial semantic embedding as input, and the neighborhood structure information is accumulated through a graph volume to obtain the structure embedding of the nodes; Step S22, mapping distribution parameters, namely mapping the