Search

CN-121980018-A - Knowledge base-based dynamic decision intelligent question-answering method and device

CN121980018ACN 121980018 ACN121980018 ACN 121980018ACN-121980018-A

Abstract

The invention discloses a knowledge base-based dynamic decision intelligent question-answering method and device. After preliminary retrieval, the method calculates decision indexes by acquiring state parameters such as knowledge base scale, query result distinction, system load and the like so as to dynamically select whether to start a second retrieval model with expensive calculation for reordering. If fine discharge is not needed, the preliminary result is directly adopted. And finally, inputting the optimized document into a large language model to generate an answer. The invention effectively balances the system precision and efficiency and obviously improves the resource utilization rate while ensuring the answer quality through self-adaptive path selection.

Inventors

  • WAN XUEFENG
  • WU JIAN
  • Dong Chenni
  • ZHAO MIN
  • HUO MEIRU
  • HAN CHAO
  • ZHANG JIANLIANG
  • DANG XIAOYAN
  • HAO XIAOWEI
  • LI YANG

Assignees

  • 国网山西省电力有限公司信息通信分公司

Dates

Publication Date
20260505
Application Date
20251126

Claims (10)

  1. 1. The knowledge base-based dynamic decision intelligent question-answering method is characterized by comprising the following steps of: step 1, receiving a target problem input by a user; Step 2, performing preliminary retrieval on the target problem in a knowledge base through a first retrieval model to obtain a candidate set comprising a plurality of candidate document fragments and corresponding preliminary relevance scores; Step 3, obtaining at least one system state parameter reflecting the current state of the system, and judging whether the candidate set needs to be subjected to deep analysis processing based on the system state parameter; Step 4, if the decision index is larger than the preset threshold, selecting a first processing path, starting a computationally intensive second retrieval model to reorder the candidate set to generate an optimized fragment set, if the decision index is not larger than the threshold, selecting a second processing path, and directly selecting fragments from the candidate set based on the preliminary relevance score to generate the optimized fragment set; and step 5, combining the optimized fragment set and the target problem, and inputting a large language model to generate and output a final answer.
  2. 2. The method according to claim 1, wherein the first search model in the step 2 adopts a mixed search strategy, and the output preliminary relevance score thereof merges a semantic similarity score based on a text vector, a text relevance score based on keyword matching, and an information timeliness score and/or a source authority score based on document metadata, wherein the mixed weight of the semantic similarity score and the text relevance score is a fixed preset super-parameter, or the mixed weight is a variable dynamically output by a query intention analyzer according to the target problem characteristic.
  3. 3. The method according to claim 1 or 2, wherein the system state parameter in step 2 at least includes a total number of document fragments contained in a knowledge base for characterizing complexity of the knowledge base, a distribution feature of each fragment preliminary relevance score in the candidate set for characterizing a distinction degree of a preliminary search result, and a current operation load state of the system for characterizing current resource availability of the system.
  4. 4. The method of claim 3 wherein the calculation of the decision index in step 3 integrates a knowledge base scale influence factor that increases as the total number of document segments contained in the knowledge base increases, a query discrimination influence factor that is inversely proportional to the degree of dispersion of the preliminary relevance score distribution for each segment in the candidate set, and a system load influence factor that is inversely proportional to the current operational load state of the system, and an interaction term of the knowledge base scale influence factor with the query discrimination influence factor.
  5. 5. The method of claim 1 or 2, wherein the system state parameters in step 2 further comprise average score of preliminary recall set, variance of preliminary recall set score, number of query terms and query type.
  6. 6. The method of claim 5, wherein the calculating of the decision index in step 3 includes organizing the system state parameters into a feature vector and inputting it into a pre-trained machine learning model to dynamically output the decision index.
  7. 7. The method of claim 1, wherein the second search model in step 4 is a Learning-to-Rank model, and wherein the input features of the Learning-to-Rank model include a deep semantic relevance score calculated by a cross encoder model, the preliminary relevance score, and features derived from document metadata.
  8. 8. The method according to claim 1, characterized in that in step 4, the process of selecting the segments directly from the candidate set based on the preliminary relevance score, in particular sorting from high to low and intercepting a predetermined number of top document segments according to the relevance score.
  9. 9. The method according to claim 1, wherein the step 5 of combining the optimized sliced collection with the target question includes organizing the contents of the sliced together with the target question into a structured prompt and inputting the structured prompt into a large language model.
  10. 10. A knowledge base-based dynamic decision intelligent question-answering device, comprising: the receiving module is used for receiving the target problem input by the user; The preliminary retrieval module is used for carrying out preliminary retrieval on the target problem in a knowledge base through a first retrieval model to obtain a candidate set comprising a plurality of candidate document fragments and corresponding preliminary relevance scores; The decision module is used for acquiring at least one system state parameter reflecting the current state of the system and judging whether the candidate set needs to be subjected to deep analysis processing based on the system state parameter; The reordering module is used for selecting a first processing path if the decision index is larger than the preset threshold value and the candidate set is judged to need to be subjected to deep analysis processing, and starting a second computationally intensive retrieval model to reorder the candidate set to generate an optimized fragment set; if the decision index is not greater than the threshold, judging that the candidate set is not required to be subjected to deep analysis, selecting a second processing path, wherein fragments are directly selected from the candidate set based on the preliminary relevance score so as to generate the optimized fragment set; And the generation module is used for combining the optimized fragment set and the target problem, inputting a large language model, and generating and outputting a final answer.

Description

Knowledge base-based dynamic decision intelligent question-answering method and device Technical Field The invention belongs to the field of intelligent question and answer, and particularly relates to a knowledge base-based dynamic decision intelligent question and answer method and device. Background With the rapid development of Large Language Models (LLM), search enhancement generation (RETRIEVAL-AugmentedGeneration, RAG) technology has become the dominant paradigm for building domain-specific knowledge question-answering systems. Conventional RAG systems typically follow a fixed "search-generate" two-step procedure by first retrieving document snippets from a knowledge base that are relevant to the user's question and then providing those snippets as context to a large language model to generate an answer. However, the prior art solutions have the following limitations: 1. Contradiction between retrieval accuracy and efficiency in order to ensure recall speed, the retrieval phase typically employs an efficient algorithm based on vector similarity (e.g., HNSW), which is known as "coarse recall". However, when dealing with large-scale, complex-content knowledge bases, the results of coarse recall often contain a large number of semantically related but not optimal "noisy" documents of answers, which can seriously affect the accuracy of the final generated answer. The introduction of a precision alignment model (such as a cross encoder) with larger calculation amount can significantly improve the retrieval precision, but the system delay and the operation cost can be greatly increased, so that precision alignment is started at the same time for all queries, and unnecessary resource waste is caused. 2. The existing system usually adopts a fixed processing flow, and cannot dynamically adjust strategies according to the size of a knowledge base, the characteristics of user inquiry or the running state of the system. For example, for a small-scale, content-refined knowledge base, a simple coarse recall may be sufficient. Conversely, for a large and heterogeneous knowledge base, the fine-pitch step is critical. The solidified architecture makes the performance and cost of the system in different application scenarios not optimally balanced. 3. Ignoring the quality of the information sources, traditional search models focus mainly on the relevance of the content level, and neglecting metadata information such as authority or timeliness of knowledge sources, which may lead to systems referencing outdated or unreliable information to generate answers. Disclosure of Invention In order to solve the technical problems, the invention provides a knowledge base-based dynamic decision intelligent question-answering method, which comprises the following steps: step 1, receiving a target problem input by a user; Step 2, performing preliminary retrieval on the target problem in a knowledge base through a first retrieval model to obtain a candidate set comprising a plurality of candidate document fragments and corresponding preliminary relevance scores; Step 3, obtaining at least one system state parameter reflecting the current state of the system, judging whether the candidate set needs to be subjected to deep analysis processing based on the system state parameter, calculating a decision index according to the system state parameter, comparing the decision index with a preset threshold value, and determining the necessity of the deep analysis processing; Step 4, if the decision index is larger than the preset threshold, selecting a first processing path, wherein a computationally intensive second retrieval model is started to reorder the candidate set to generate an optimized fragment set, and if the decision index is not larger than the threshold, selecting a second processing path, wherein fragments are directly selected from the candidate set based on the preliminary relevance score to generate the optimized fragment set; and step 5, combining the optimized fragment set and the target problem, and inputting a large language model to generate and output a final answer. The first search model in the step 2 adopts a mixed search strategy, and the output preliminary relevance score of the mixed search strategy fuses a semantic similarity score based on a text vector, a text relevance score based on keyword matching and an information timeliness score and/or source authority score based on document metadata, wherein the mixed weight of the semantic similarity score and the text relevance score is a fixed preset super parameter, or the mixed weight is a variable dynamically output by a query intention analyzer according to the target problem characteristic. Particularly, the system state parameters in the step 2 at least comprise the total number of document fragments contained in a knowledge base and used for representing the complexity of the knowledge base, the distribution characteristics of the preliminary relevance sc