Search

CN-121145812-B - Text style conversion method and device based on semantic and style double-condition injection

CN121145812BCN 121145812 BCN121145812 BCN 121145812BCN-121145812-B

Abstract

The invention provides a text style conversion method and a text style conversion device based on semantic and style double-condition injection, which are characterized in that a pre-trained semantic extraction model is used for extracting semantic vectors from a user source text and retrieving standard style vectors of target types from a built-in standard style library, the semantic vectors and the standard style vectors are sent into a text style migration model based on ChatGLM through low-rank self-adaptive fine tuning of a single style sample set based on a double-layer injection mechanism on the basis of a task instruction, and target texts which are consistent with the source text semantics and are in style fit are generated through autoregressive. The text style migration model only uses the same style sample in the fine tuning stage, applies a random mask to the sample to obtain a incomplete semantic vector, then splices the task instruction and the style vector to reconstruct the complete text of the model, judges the category of the reconstruction result through a style identifier, and synthesizes semantic deviation, style deviation and reconstruction error to construct multiple loss feedback updating. The method effectively guides the model to generate the high-quality text conforming to the target style, and realizes the balance between semantic preservation and style conversion.

Inventors

  • HAN SHUHUAN

Assignees

  • 北京邮电大学

Dates

Publication Date
20260505
Application Date
20250912
Priority Date
20250812

Claims (10)

  1. 1. A text style conversion method based on semantic and style double-condition injection is characterized by comprising the following steps: Acquiring source text and target text style provided by a user, extracting a source text semantic feature vector with a first set dimension from the source text by adopting a preset semantic extraction model, and inquiring a predefined style database based on the target text style to acquire a canonical style vector with a second set dimension; Inputting a task instruction based on natural language, the source text semantic feature vector and the standard style vector into a preset text style migration model through a double-layer injection mechanism to output a target text which is consistent with the source text semantic and accords with the target text style type, wherein the task instruction is used for indicating to execute a text style conversion task, the double-layer injection mechanism comprises the steps of injecting the semantic feature vector into one layer 1/3 to 1/2 of the front of the text style migration model, and injecting the standard style vector into one layer 1/3 of the rear of the text style migration model; The text style migration model is obtained by carrying out low-rank self-adaptive fine tuning on a ChatGLM large language model by adopting a single-style text sample set, each sample of the single-style text sample set comprises a task instruction based on natural language, a sample text and a corresponding sample text style label, the sample text subjected to random damage is extracted by adopting the semantic extraction model, the random damage comprises mask marks, word deletion and similar character replacement implemented according to a preset proportion, the sample text style database is queried based on the sample text style label to obtain a sample style vector, the task instruction, the incomplete semantic feature vector and the sample style vector are input into the text style migration model through the double-layer injection mechanism to execute an autoregressive task to output a reconstructed text, the reconstructed text style is identified based on a preset text style identification model, text reconstruction deviation loss, semantic deviation loss and text style deviation loss and joint loss are calculated based on the reconstructed text and the sample text, and the text migration model is subjected to low-rank self-adaptive parameter updating by adopting a joint loss function.
  2. 2. The text style conversion method based on semantic and style dual-condition injection according to claim 1, wherein the semantic extraction model adopts a BERT model based on a transform architecture and performs 768-dimensional semantic feature vector extraction based on CLS pooling, the text style migration model is ChatGLM model, a rank parameter is set to 8, a scaling parameter is set to 32, a dropout rate is set to 0.1 in the low-rank adaptive parameter updating process, and main parameters of the ChatGLM model are frozen and parameter updating is performed for a query layer and a key value layer.
  3. 3. The text style conversion method based on semantic and style double conditional injection according to claim 1, wherein the double injection mechanism comprises injecting the semantic feature vector into one of 1/3 to 1/2 before the text style migration model, and injecting the canonical style vector into one of 1/3 after the text style migration model, comprising: The semantic feature vector of the source text is injected into a 6 th layer hidden state of a ChatGLM model after being mapped by a projection layer, the canonical style vector is injected into an 18 th layer hidden state of a ChatGLM model after being mapped by the projection layer, and the fusion proportion of the injected vector and the original hidden state is controlled by a gating mechanism.
  4. 4. The text style conversion method based on semantic and style double conditional injection according to claim 1, wherein the constructing step of the predefined style database comprises: Inputting representative samples of a plurality of text styles into a pre-trained style encoder to output representative style vectors of the representative samples, wherein the style encoder comprises a BERT model and a projection mapping layer and is used for outputting the representative style vectors of 256 dimensions; And calculating an average value of a plurality of representative style vectors corresponding to each text style to obtain the standard style vector corresponding to the text style.
  5. 5. The text style conversion method based on semantic and style bi-conditional injection according to claim 1, wherein the text style recognition model comprises a text style encoder and a text style classifier, the text style encoding is composed of a BERT model and a projection mapping layer, the projection mapping layer comprises a linear layer, a ReLU activation function, a Dropout layer and a LayerNorm layer, the projection mapping layer maps 768-dimensional BERT output into 256-dimensional style vectors, the text style classifier is composed of a plurality of single-wire connection layers and maps 256-dimensional style vectors into a category logits of a style category number, and the pre-training step of the text style recognition model comprises: acquiring a text style classification training data set containing a plurality of style categories and a plurality of samples, wherein each sample contains a sample text and a corresponding style classification label; And constructing a style classification loss based on the deviation of the style prediction and the style classification labels, constructing an intra-style comparison loss by the sample style vectors corresponding to each sample text so as to reduce the difference between similar styles and expand the difference between different types of styles, and updating parameters of the text style recognition model by combining the style classification loss and the intra-style comparison loss.
  6. 6. The text style conversion method based on semantic and style double conditional injection according to claim 5, wherein the style classification loss is calculated as: ; Wherein, the Representing the loss of classification of the style in question, In order for the cross-entropy loss to occur, The prediction of the style is represented by a model, Representing the style classification tag; The calculation formula of the contrast loss in the style is as follows: Wherein, the Representing the loss of contrast in the style in question, An anchor style vector representing anchor samples selected within a batch, Representing the jth positive sample style vector within the batch that is the same as the anchor sample style, Representing a set of all positive samples; representing a kth negative sample style vector within the batch that is different from the anchor sample style, Representing a set of all negative samples, sim represents a cosine similarity calculation, exp represents an exponential function, Is a temperature super parameter; And in the process of carrying out parameter updating on the text style recognition model by combining the style classification loss and the intra-style comparison loss, a first combined loss calculation formula is adopted as follows: ; Wherein, the Representing the first loss of said association and, And Is a weight coefficient.
  7. 7. The text style conversion method based on semantic and style bi-conditional injection of claim 5, wherein parameter updating the text style migration model based on semantic deviation, style deviation, and text reconstruction deviation construction penalty of the reconstructed text style and the sample text comprises: respectively extracting semantic vectors from the reconstructed text style and the sample text by adopting a preset semantic extraction model, and calculating semantic deviation loss based on cosine similarity; calculating cross entropy between the reconstructed text style and the sample text style label as style type deviation loss; calculating the cross entropy of the text style migration model on the prediction distribution of each word element of the reconstructed text and the real distribution of each word element in the sample text as text reconstruction deviation loss; And carrying out weighted summation on the semantic deviation loss, the style deviation loss and the text reconstruction deviation loss to construct a second joint loss, and carrying out parameter updating on the text style migration model by minimizing the second joint loss.
  8. 8. A text style conversion device incorporating semantic and style vectors, comprising a processor, a memory and computer program/instructions stored on the memory, wherein the processor is configured to execute the computer program/instructions, which when executed, implement the steps of the method of any one of claims 1 to 7.
  9. 9. A computer readable storage medium having stored thereon a computer program/instruction which when executed by a processor performs the steps of the method according to any of claims 1 to 7.
  10. 10. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 7.

Description

Text style conversion method and device based on semantic and style double-condition injection Technical Field The invention relates to the technical field of natural language processing and deep learning, in particular to a text style conversion method and device based on semantic and style double-condition injection. Background Language style conversion is an important task in the field of natural language processing, aimed at converting a piece of text from one style to another while keeping the core semantics of the original text unchanged. For example, a piece of ordinary narrative text is converted into a poetry style, a gorgeous style, a news style, or the like. With the development of deep learning technology, especially the appearance of pre-trained language models, new technical possibilities are provided for language style conversion. Existing language style conversion methods mainly include rule-based methods, statistical-based methods, and neural network-based methods. The method based on the rule realizes style conversion through a manually defined conversion rule, but has complex rule design and limited coverage, the statistical method based on the statistics learns a style conversion mode through a statistical model, but is difficult to capture deep semantics, and the neural network based method directly learns style conversion mapping through a deep learning model, and has the following problems that the semantic preservation and the style conversion are difficult to balance, the converted text possibly deviates from original text semantics, style characteristics are difficult to quantitatively represent, the style conversion is not accurate enough, and the quality of the generated text is unstable and may be in a non-fluent or incoherent condition. Thus, a new text style conversion scheme is needed. Disclosure of Invention In view of this, the embodiment of the invention provides a text style conversion method and device based on semantic and style double-condition injection, so as to eliminate or improve one or more defects existing in the prior art, and solve the problems of poor text style conversion effect, missing semantic reservation and poor generated text quality in the prior art. One aspect of the present invention provides a text style conversion method based on semantic and style double-condition injection, the method comprising the steps of: Acquiring source text and target text style provided by a user, extracting a source text semantic feature vector with a first set dimension from the source text by adopting a preset semantic extraction model, and inquiring a predefined style database based on the target text style to acquire a canonical style vector with a second set dimension; Inputting a task instruction based on natural language, the source text semantic feature vector and the standard style vector into a preset text style migration model through a double-layer injection mechanism to output a target text which is consistent with the source text semantic and accords with the target text style type, wherein the task instruction is used for indicating to execute a text style conversion task, the double-layer injection mechanism comprises the steps of injecting the semantic feature vector into one layer 1/3 to 1/2 of the front of the text style migration model, and injecting the standard style vector into one layer 1/3 of the rear of the text style migration model; The text style migration model is obtained by carrying out low-rank self-adaptive fine tuning on a ChatGLM large language model by adopting a single-style text sample set, each sample of the single-style text sample set comprises a task instruction based on natural language, a sample text and a corresponding sample text style label, the sample text subjected to random damage is extracted by adopting the semantic extraction model, the random damage comprises mask marks, word deletion and similar character replacement implemented according to a preset proportion, the sample text style database is queried based on the sample text style label to obtain a sample style vector, the task instruction, the incomplete semantic feature vector and the sample style vector are input into the text style migration model through the double-layer injection mechanism to execute an autoregressive task to output a reconstructed text, the reconstructed text style is identified based on a preset text style identification model, text reconstruction deviation loss, semantic deviation loss and text style deviation loss and joint loss are calculated based on the reconstructed text and the sample text, and the text migration model is subjected to low-rank self-adaptive parameter updating by adopting a joint loss function. In some embodiments, the semantic extraction model adopts a BERT model based on a transducer architecture, 768-dimensional semantic feature vector extraction is performed based on CLS pooling, the text style migration model