CN-116975235-B - Natural language answer generation method, device and medium based on multi-level replication and perception loss

CN116975235BCN 116975235 BCN116975235 BCN 116975235BCN-116975235-B

Abstract

The invention discloses a natural language answer generation method, a device and a medium based on multi-level replication and perception loss, wherein the method comprises the following steps: and respectively inputting the natural language questions and the triples into corresponding coding layers according to the pre-acquired natural language questions and triples to obtain vector representations of the natural language questions and the triples. The natural language answer generating method, the device and the medium based on multi-level copying and perception loss take natural language answers in an intelligent QA system as research objects, and a decoder based on a multi-level copying and predicting mechanism is provided for generating nonsensical responses and questions and answers mismatch problems in the existing method, and relevant semantic units in the questions and triples are copied at the same time, and common vocabularies are predicted from a vocabulary, so that natural language answer sequences conforming to grammar and context are generated according to user requirements, and theoretical basis is laid for complex multi-round conversations in the next step.

Inventors

ZHAO FEN
ZHANG YUZHOU
XIAO WENJIE
QI ZHENGDONG

Assignees

南京晓庄学院

Dates

Publication Date: 20260505
Application Date: 20230731

Claims (10)

1. A natural language answer generation method based on multi-level replication and perception loss is characterized by comprising the following steps: according to the pre-acquired natural language questions and triples, respectively inputting the natural language questions and triples into corresponding coding layers to obtain vector representations of the natural language questions and triples; Obtaining a semantic unit set and a triplet entity set of a problem in a copying mode based on a pre-designed decoder according to the obtained vector representation; generating common words in a natural language answer sequence based on a prediction mechanism according to a pre-acquired vocabulary, wherein the common words at least comprise verbs, prepositions, definite articles and indefinite articles; generating a complete natural language answer sequence based on the obtained semantic unit set, the entity set and the common words; optimizing the generation of the natural language answer sequence based on the answer perception loss according to the obtained natural language answer sequence, and generating the natural language answer sequence conforming to grammar; Generating a target answer sequence which accords with the context and is matched with the question based on the question perception loss according to the obtained natural language answer sequence which accords with the grammar; The pre-designed decoder comprises a problem replication mechanism module, a triplet replication mechanism module and a prediction mechanism module.
2. The natural language answer generation method based on multi-level replication and perceptual loss as defined in claim 1, wherein the obtaining method of the semantic unit set comprises: Copying relevant semantic units from the source questions based on a question copying mechanism in a pre-designed decoder according to the obtained vector representation of the natural language questions; Based on the copied related semantic units, the related semantic units are used as a semantic unit set.
3. The natural language answer generation method based on multi-level replication and perceptual loss as defined in claim 1, wherein the obtaining method of the entity set comprises: Copying related entities from triples based on a triplet copying mechanism in a pre-designed decoder according to the obtained vector representation of the triples; The relevant entity based on replication is taken as an entity set.
4. The natural language answer generation method based on multi-level replication and perceptual loss according to claim 1, wherein the generation method of common words in the natural language answer sequence comprises: the prediction mechanism module based on the decoder predicts the common words by the pre-acquired vocabulary, judges whether the scores of the predicted common words meet a preset threshold value, selects the common words with the scores meeting the preset threshold value as the common words in the natural language answer sequence, wherein, The decoder predicts the scoring function of the commonly used words from the vocabulary based on the prediction mechanism as: φ predict (w t ＝v i )＝v i W predict [d t ,c Qt ,c Tt ] Wherein the pre-obtained vocabulary includes V Q ∪V T ,V＝{v 1 ,v 2 ,...,v n-1 ,v n } { OOVW }, OOVW represents any word-library-external word, V Q represents a semantic unit set of a problem, V T represents an entity set of a triplet, Φ predict (w t ＝v i ) represents a scoring function of a prediction mechanism for predicting common words from the vocabulary, predict represents a prediction mechanism for common words, W t represents a natural language answer sequence generated at time t, V i represents a word vector of an output layer, W predict represents a weight matrix of the prediction mechanism, and c Qt and c Tt represent context vectors selectively read from short-term memories of the problem h Q and the triplet fact h T , respectively, at time step t.
5. The method for generating natural language answers based on multi-level replication and perceptual loss of claim 1, wherein the probability of the generated complete natural answer sequence is required to satisfy a predetermined threshold, wherein, The probability language model of the generated complete natural answer sequence is expressed as: Wherein w t-1 represents a natural language answer sequence generated at time t-1, P (w t |d t ,w t-1 ,h Q ,h T ) is a probabilistic language model for generating any natural language answer sequence, w t is a generated complete natural language answer sequence, copy Q 、copy T and predict respectively represent a replication mechanism of a problem, a replication mechanism of a triplet and a prediction mechanism of a common word, d t is a hidden state of a decoder, h Q is a vector representation of the whole natural language problem, h T is a vector representation of a fact triplet, t is a time step, P d (|) represents a probabilistic language model for selecting different mechanisms, and c t represents a context vector.
6. The method for generating natural language answers based on multi-level replication and perceptual loss of claim 5, wherein said method for generating a grammar-compliant natural language answer sequence comprises: Training a natural answer sequence probability language model according to a given natural question Q, a fact triplet T and a target answer sequence W= (W 1 ,w 2 ,...,w t ,...,w |W| ); optimizing negative log likelihood through an answer perception function for a natural language answer sequence generated by the natural answer sequence probability language model to generate a natural language answer sequence conforming to grammar, wherein, The answer sense loss function is the following: Wherein, psi A_loss is an answer perception loss function, L 2 is a regularization factor, kappa is a super parameter of L 2 , W is a target answer sequence, h Q is a vector representation of the whole natural language question, h T is a vector representation of a fact triplet, and t is a time step.
7. The method for generating natural language answers based on multi-level replication and perceptual loss of claim 1, wherein said method for generating a context-compliant, question-matching target answer sequence comprises: Generating a target answer subsequence corresponding to the question type word according to the input question based on the generated natural language answer sequence conforming to the grammar; generating answer words matched with the question type words through the question perception loss function, wherein the cross entropy of the question type words and the answer words is required to be minimum; optimizing the generation of a natural language answer sequence based on the answer words to generate a target answer sequence matched with the questions and conforming to the context, wherein, The problem-aware loss function is represented by the following formula: Wherein, psi Q_loss is the problem perception loss, Is a set of question type words, Y is a set of answer words generated, The cross entropy between the question type word q n and the generated answer word w t is represented.
8. A natural language answer generation device based on multi-level replication and perceptual loss, the device comprising: the first data processing module is used for respectively inputting the natural language questions and the triples into corresponding coding layers according to the pre-acquired natural language questions and triples to obtain vector representations of the natural language questions and the triples; The second data processing module is used for obtaining a semantic unit set and a triplet entity set of the problem in a copying mode based on a pre-designed decoder according to the obtained vector representation; the third data processing module is used for generating common words in the natural language answer sequence based on a prediction mechanism according to a pre-acquired vocabulary, wherein the common words at least comprise verbs, prepositions, definite articles and indefinite articles; the fourth data processing module is used for generating a complete natural language answer sequence based on the obtained semantic unit set, the entity set and the common words; The fifth data processing module is used for optimizing the generation of the natural language answer sequence based on the answer perception loss according to the obtained natural language answer sequence and generating the natural language answer sequence conforming to grammar; and the target answer sequence generation module is used for generating a target answer sequence which accords with the context and is matched with the question based on the question perception loss according to the obtained natural language answer sequence which accords with the grammar.
9. A natural language answer generation device based on multi-level copying and perception loss is characterized by comprising a processor and a storage medium; The storage medium is used for storing instructions; the processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 7.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

Description

Natural language answer generation method, device and medium based on multi-level replication and perception loss Technical Field The invention relates to a natural language answer generation method, device and medium based on multi-level replication and perception loss, belonging to the technical field of natural language processing. Background Rapidly evolving artificial intelligence techniques have become a hotspot for social concern. The intelligent question-answering (Question Answering, QA) system is an important branch of artificial intelligence technology, and has become an important research topic of interest in academia and industry in recent years because it can capture user intention more accurately, understand natural language questions of users, and return concise and accurate answers. Most existing QA systems reason about the correct physical answer, however in real-world environments users like to answer natural language questions in a more natural way, while existing methods also have problems of generating nonsensical responses and questions-answers mismatch and generating natural language answers mismatch with the input questions. Therefore, in order to solve the above technical problems, a method, a device and a medium for generating natural language answers based on multi-level replication and perception loss are needed. Disclosure of Invention The invention aims to overcome the defects in the prior art, and provides a natural language answer generation method, device and medium based on multi-level replication and perception loss, aiming at the problems of nonsensical response and mismatching of questions and answers generated by the prior method, a decoder based on a multi-level replication and prediction mechanism is provided, related semantic units in the problems and triples are replicated, common words are predicted from a vocabulary, natural language answers meeting grammar are finally generated, and the perception loss of the problems and the perception loss of the answers are combined, so that natural language answer sequences meeting grammar and context are generated according to the requirements of users, and theoretical basis is laid for complex multi-round conversations in the next step. In order to achieve the above purpose, the invention is realized by adopting the following technical scheme: in a first aspect, the present invention provides a natural language answer generation method based on multi-level replication and perceptual loss, including: according to the pre-acquired natural language questions and triples, respectively inputting the natural language questions and triples into corresponding coding layers to obtain vector representations of the natural language questions and triples; Obtaining a semantic unit set and a triplet entity set of a problem in a copying mode based on a pre-designed decoder according to the obtained vector representation; generating common words in a natural language answer sequence based on a prediction mechanism according to a pre-acquired vocabulary, wherein the common words at least comprise verbs, prepositions, definite articles and indefinite articles; generating a complete natural language answer sequence based on the obtained semantic unit set, the entity set and the common words; optimizing the generation of the natural language answer sequence based on the answer perception loss according to the obtained natural language answer sequence, and generating the natural language answer sequence conforming to grammar; Generating a target answer sequence which accords with the context and is matched with the question based on the question perception loss according to the obtained natural language answer sequence which accords with the grammar; The pre-designed decoder comprises a problem replication mechanism module, a triplet replication mechanism module and a prediction mechanism module. Further, the method for obtaining the semantic unit set comprises the following steps: Copying relevant semantic units from the source questions based on a question copying mechanism in a pre-designed decoder according to the obtained vector representation of the natural language questions; Based on the copied related semantic units, the related semantic units are used as a semantic unit set. Further, the method for obtaining the entity set comprises the following steps: Copying related entities from triples based on a triplet copying mechanism in a pre-designed decoder according to the obtained vector representation of the triples; The relevant entity based on replication is taken as an entity set. Further, the method for generating the common words in the natural language answer sequence comprises the following steps: the prediction mechanism module based on the decoder predicts the common words by the pre-acquired vocabulary, judges whether the scores of the predicted common words meet a preset threshold value, selects the common words with the sc