CN-121979984-A - Travel intelligent question-answering method and system based on large language model

CN121979984ACN 121979984 ACN121979984 ACN 121979984ACN-121979984-A

Abstract

The invention discloses a travel intelligent question-answering method and system based on a large language model, and relates to the technical field of semantic analysis; the method comprises the steps of preprocessing query texts to obtain embedded vectors, mapping map entities into third vectors, screening and constructing a candidate library, extracting context features and clustering to obtain query vectors, splicing candidate entities into knowledge vectors, fusing to obtain knowledge semantic vectors, generating semantic features by combining the context features, inputting a language model to output a prediction result, extracting the context features and clustering to generate the query vectors, generating the query vectors representing core query intention, avoiding semantic dispersion, fusing the knowledge semantic vectors of the candidate entity vectors and the query vectors, making up the defect that pure text semantics lack of structural knowledge support, combining the context features to generate semantic feature vectors, inputting the language model to output the prediction result, and solving the problem that ambiguity occurs to user replies due to the fact that knowledge behind words and context background information are difficult to utilize.

Inventors

LU HAO
Li Shaokuo
WEI SHAOKANG

Assignees

宋城独木桥网络有限公司

Dates

Publication Date: 20260505
Application Date: 20260115

Claims (10)

1. A travel intelligent question-answering method based on a large language model, which is characterized by comprising the following steps: Acquiring inquiry text data input by tourists, and preprocessing the inquiry text data to obtain a first embedded vector and a second embedded vector, wherein the first embedded vector is an embedded vector of a single word in a context text of a target sentence, and the second embedded vector is an embedded vector of a single word in the target sentence; obtaining a travel knowledge graph, mapping each entity in the travel knowledge graph to obtain a third embedded vector, screening the second embedded vector and the third embedded vector, and constructing a candidate entity memory bank; Extracting context semantic features from the first embedded vector to obtain a context feature vector, and performing density clustering on the context feature vector to generate a query vector; splicing the candidate entity vector in the candidate entity memory library with the query vector to obtain a knowledge vector, and fusing the second embedded vector with the knowledge vector to obtain a knowledge semantic vector; and carrying out preset operation according to the context feature vector and the knowledge semantic vector to obtain a semantic feature vector, and inputting the semantic feature vector into a language model to output a semantic prediction result.
2. The large language model based travel intelligent question-answering method according to claim 1, wherein the steps of screening the second embedded vector and the third embedded vector and constructing a candidate entity memory library include: mapping each entity in the knowledge graph into a low-dimensional real-valued vector through TransE model to obtain an embedded vector of a single entity in the knowledge graph, and defining the embedded vector as a third embedded vector; And screening the third embedded vector meeting the condition according to the similarity and a similarity threshold value to serve as a candidate entity embedded vector by calculating the similarity of the second embedded vector and the third embedded vector, and combining all the candidate entity embedded vectors into a candidate entity memory bank.
3. The large language model based travel intelligent question-answering method according to claim 1, wherein density clustering the context feature vectors to generate query vectors comprises: capturing the text front-back semantic dependency relationship through a bidirectional attention mechanism of a transducer, and generating a context feature vector; calculating the concentration of K neighbor features around each context feature vector and the distance index of each context feature vector; Calculating the joint scores of each context feature vector according to the concentration degree and the distance index, and selecting the first M context feature vectors as potential clustering centers by descending arrangement of the joint scores; And classifying the feature vectors with the semantic similarity smaller than the threshold value in the K neighbors of each potential cluster center into the same cluster, calculating the average value of all the context feature vectors in the cluster to obtain a cluster center, and taking the cluster center as a query vector.
4. The method of claim 1, wherein fusing the second embedded vector with the knowledge vector to obtain a knowledge semantic vector, comprises: Splicing each candidate entity vector in the candidate entity memory library with the query vector to obtain a first spliced vector; Inputting the first spliced vectors into two full-connection layers, calculating the attention weight of each candidate entity by using a Softmax function after the activation function is activated, and carrying out weighted summation on all the first spliced vectors according to the attention weight to obtain a knowledge vector; And splicing each second embedded vector with the knowledge vector to obtain an enhanced second embedded vector, and inputting the enhanced second embedded vector into the BERT model to output a knowledge semantic vector.
5. The method for travel intelligent question-answering based on a large language model according to claim 1, wherein the step of performing a preset operation according to the context feature vector and the knowledge semantic vector to obtain the semantic feature vector comprises the steps of: Inputting the context feature vector and the knowledge semantic vector into a full connection layer respectively, and activating through a tanh function to obtain an enhanced context feature vector and an enhanced knowledge semantic vector respectively; Calculating semantic matrixes of the enhanced context feature vectors and the enhanced knowledge semantic vectors through matrix multiplication, and respectively outputting a first attention score matrix and a second attention score matrix for the semantic matrixes and transposed matrixes thereof through Softmax functions; Weighting the enhancement knowledge semantic vector according to the first attention score matrix to obtain a first correction vector, and splicing the first correction vector and the enhancement context feature vector to obtain a second spliced vector; and weighting the second spliced vector according to the second attention score matrix to obtain a second corrected vector, splicing the second corrected vector with the enhanced knowledge semantic vector, and inputting the spliced second corrected vector and the enhanced knowledge semantic vector into the BiLSTM module to obtain the semantic feature vector.
6. The travel intelligent question-answering system based on the large language model is characterized by comprising a preprocessing module, a screening module, a clustering module, a fusion module and a prediction module, wherein: the preprocessing module is used for acquiring query text data input by tourists, preprocessing the query text data to obtain a first embedded vector and a second embedded vector, wherein the first embedded vector is an embedded vector of a single word in a context text of a target sentence, and the second embedded vector is an embedded vector of a single word in the target sentence; the screening module is used for acquiring a travel knowledge graph, mapping each entity in the travel knowledge graph to obtain a third embedded vector, screening the second embedded vector and the third embedded vector, and constructing a candidate entity memory bank; the clustering module is used for extracting context semantic features from the first embedded vector to obtain a context feature vector, and performing density clustering on the context feature vector to generate a query vector; the fusion module is used for splicing the candidate entity vectors in the candidate entity memory library with the query vectors to obtain knowledge vectors, and fusing the second embedded vectors with the knowledge vectors to obtain knowledge semantic vectors; The prediction module is used for carrying out preset operation according to the context feature vector and the knowledge semantic vector to obtain a semantic feature vector, and inputting the semantic feature vector into the language model to output a semantic prediction result.
7. The travel intelligent question-answering system based on big language model according to claim 6, wherein the screening includes a mapping module and a combining module, wherein: The mapping module is used for mapping each entity in the knowledge graph into a low-dimensional real-value vector through a TransE model to obtain an embedded vector of a single entity in the knowledge graph, and defining the embedded vector as a third embedded vector; The combination module is used for screening the third embedded vector meeting the condition according to the similarity and the similarity threshold value to serve as a candidate entity embedded vector by calculating the similarity of the second embedded vector and the third embedded vector, and combining all the candidate entity embedded vectors into a candidate entity memory bank.
8. The travel intelligent question-answering system based on a large language model according to claim 6, wherein the clustering module comprises a capturing module, a first computing module, a second computing module, a cluster generating module, wherein: the capturing module is used for capturing the semantic dependency relationship before and after the text through a bidirectional attention mechanism of the transducer and generating a context feature vector; The first calculation module is used for calculating the density of K neighbor features around each context feature vector and the distance index of each context feature vector; The second calculation module is used for calculating the joint scores of each context feature vector according to the concentration degree and the distance index, and selecting the previous M context feature vectors as potential clustering centers by descending order of the joint scores; And the cluster generation module is used for classifying the feature vectors with the semantic similarity smaller than the threshold value in the K neighbors of each potential cluster center into the same cluster, calculating the average value of all the context feature vectors in the cluster to obtain a cluster center, and taking the cluster center as a query vector.
9. The travel intelligent question-answering system based on a large language model according to claim 6, wherein the fusion module comprises a first splicing module, a third calculation module, and an enhancement module, wherein: The first splicing module is used for splicing each candidate entity vector in the candidate entity memory library with the query vector to obtain a first spliced vector; the third calculation module is used for inputting the first spliced vectors into two full-connection layers, calculating the attention weight of each candidate entity by using a Softmax function after the activation function is activated, and carrying out weighted summation on all the first spliced vectors according to the attention weight to obtain a knowledge vector; The enhancement module is used for splicing each second embedded vector with the knowledge vector to obtain an enhanced second embedded vector, and inputting the enhanced second embedded vector into the BERT model to output the knowledge semantic vector.
10. The travel intelligent question-answering system based on a large language model according to claim 6, wherein the prediction module comprises an activation module, a fourth calculation module, a correction module, a weighting module, wherein: the activation module is used for inputting the context feature vector and the knowledge semantic vector into the full-connection layer respectively, and activating the context feature vector and the knowledge semantic vector respectively through the tanh function to obtain an enhanced context feature vector and an enhanced knowledge semantic vector; The fourth calculation module is used for calculating semantic matrixes of the enhanced context feature vectors and the enhanced knowledge semantic vectors through matrix multiplication, and outputting a first attention score matrix and a second attention score matrix for the semantic matrixes and transposed matrixes thereof through a Softmax function respectively; The correction module is used for weighting the enhancement knowledge semantic vector according to the first attention score matrix to obtain a first correction vector, and splicing the first correction vector with the enhancement context feature vector to obtain a second spliced vector; The weighting module is used for weighting the second spliced vector according to the second attention score matrix to obtain a second corrected vector, splicing the second corrected vector with the enhanced knowledge semantic vector, and inputting the spliced second corrected vector and the enhanced knowledge semantic vector to the BiLSTM module to obtain the semantic feature vector.

Description

Travel intelligent question-answering method and system based on large language model Technical Field The invention belongs to the technical field of semantic analysis, and particularly relates to a travel intelligent question-answering method and system based on a large language model. Background The intelligent question-answering system in the tourism field is an intelligent interaction platform constructed based on natural language processing, knowledge graph, big data analysis and other technologies, and the core is to realize the precision, real-time and individuation of tourism consultation services through technology energization. The system integrates multidimensional travel data such as scenic spot information, traffic ticketing, accommodation catering, trip planning, emergency help seeking and the like, supports multi-channel interaction such as voice, characters, images and the like, can quickly understand natural language requirements of tourists, accurately solves high-frequency problems such as scenic spot open time, ticket booking flow, route connection scheme and the like, and can generate customized suggestions according to travel preference, time budget and the like of users. The intelligent tourist management system breaks through the space-time limit of the traditional manual consultation, simplifies the information acquisition process, improves the travel experience of tourists, provides data support for tourist practitioners to optimize service layout and mine user requirements, and is a core interaction hub for connecting the tourists with service resources in intelligent tourist ecology. The prior publication number is CN119621902A, and discloses a question-answering method and device based on a large language model, wherein the method comprises the steps of obtaining query text which is input by a user and corresponds to form data to be processed, generating prompt text corresponding to the large language model based on the query text, wherein the prompt text comprises the query text and an indication text which is used for indicating that an answer text can contain data processing instructions corresponding to the form data, inputting the prompt text into the large language model, and generating initial answer text corresponding to the query text under the guidance of the prompt text by the large language model. Although the above scheme optimizes the answer text generated by reasoning the large language model based on the query text in the intelligent dialogue system through the data processing result corresponding to the data processing instruction, in some scenes, such as the tourism field, text data output by a user may contain emotion of the user and words which are easy to cause ambiguity, the defect of structural knowledge support of the plain text may be caused, and the problems of low information retrieval efficiency and ambiguity of replies to the user are caused by difficulty in utilizing the common knowledge behind the words and context background information. Disclosure of Invention The invention aims to solve the problems that the information retrieval efficiency is low and the answer to the user is ambiguous due to the fact that the knowledge behind words and the contextual background information are difficult to use due to the fact that the pure text semantics lack of structural knowledge support in the tourism field, and provides a tourism intelligent question-answering method and system based on a large language model. In a first aspect of the present invention, a method for travel intelligent question-answering based on a large language model is first provided, the method comprising: Acquiring inquiry text data input by tourists, and preprocessing the inquiry text data to obtain a first embedded vector and a second embedded vector, wherein the first embedded vector is an embedded vector of a single word in a context text of a target sentence, and the second embedded vector is an embedded vector of a single word in the target sentence; obtaining a travel knowledge graph, mapping each entity in the travel knowledge graph to obtain a third embedded vector, screening the second embedded vector and the third embedded vector, and constructing a candidate entity memory bank; Extracting context semantic features from the first embedded vector to obtain a context feature vector, and performing density clustering on the context feature vector to generate a query vector; splicing the candidate entity vector in the candidate entity memory library with the query vector to obtain a knowledge vector, and fusing the second embedded vector with the knowledge vector to obtain a knowledge semantic vector; and carrying out preset operation according to the context feature vector and the knowledge semantic vector to obtain a semantic feature vector, and inputting the semantic feature vector into a language model to output a semantic prediction result. By respectively extracting the w