CN-122019738-A - Intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement

CN122019738ACN 122019738 ACN122019738 ACN 122019738ACN-122019738-A

Abstract

The invention belongs to the technical field of natural language processing and information retrieval, and relates to an intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement, which comprises six steps of query preprocessing, agent-based multi-tool dynamic routing, structured and semantic enhanced index construction, self-adaptive context assembly, double-layer memory management, data closed-loop and self-evolution, wherein the technical defects of semantic fracture, low retrieval accuracy, no long-term memory and lack of self-optimization of the traditional RAG system are overcome by adopting a differential segmentation strategy on text, codes and multi-mode data and combining query optimization, intelligent tool routing and a hybrid retrieval algorithm; the invention improves the code retrieval accuracy and the context utilization rate, realizes personalized long-term service and automatic operation and maintenance, and is suitable for technical research and development, code library maintenance, intelligent question-answering and other scenes.

Inventors

YANG SIEN
GUO FANRONG
ZHANG GUANGDA
WANG XU
YAN QI

Assignees

图灵人工智能研究院（南京）有限公司

Dates

Publication Date: 20260512
Application Date: 20260416

Claims (10)

1. An intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement is characterized by comprising the following steps: (1) Preprocessing the query, namely rewriting the original query of the user, extracting key entities to form metadata filtering conditions, and generating hypothetical pseudo codes for code-code mode retrieval if the query is identified to be related to code retrieval; (2) The multi-tool dynamic routing based on the Agent comprises the steps of deploying a central routing Agent, analyzing the query intention of a user, and selecting a search tool from a preset search tool set according to semantic matching degree, wherein the search tool set comprises a document search tool, a code library search tool, a FAQ search tool and a networking search tool; (3) Structuring and semantically enhancing index building: the text data processing is to divide the file according to the title level of the file preferentially, and divide the file according to the maximum character length when no clear title or the text length exceeds a preset threshold value; Code data processing, namely converting a code file into an abstract syntax Tree AST by utilizing a Tree-sitter analyzer, taking the identified high-level semantic nodes as segmentation units, extracting code meta information, and calling a large language model to generate a function description abstract for each code segmentation block; the mixed index construction, namely calculating a search score by adopting a mixed search mechanism combining vector cosine similarity with a BM25 algorithm; (4) Setting a length threshold Tmax, returning the full text when the retrieval result length L is smaller than Tmax, extracting core information of a long text through a large language model when L is larger than or equal to Tmax, extracting a skeleton of a long code, and reserving a key logic structure; (5) Double-layer memory management, namely constructing a short-term scene buffer area for storing a structured interaction log, an asynchronous memory solidification module for regularly extracting long-term information and a long-term semantic knowledge base for realizing knowledge persistence storage and retrieval; (6) Recording complete links and user feedback of each interaction, generating positive and negative sample test data sets, verifying answer consistency through regression test when the system is updated, and realizing data-driven self-optimization.
2. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement according to claim 1, wherein the key entities comprise programming language, device model, date range and technical terms, and the metadata filtering condition is realized through metadata filtering expressions of a vector library.
3. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement according to claim 1, wherein the multi-modal analysis tool comprises mineru, paddleocr-vl, the non-text format file comprises PDF and PPT, and docx format files are generated for title analysis after conversion.
4. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement according to claim 1, wherein the code meta-information comprises a file path, a file type, a module name, a method name, a start line number, an end line number and a keyword, and the code segmentation adopts a regular expression capture function annotation and a grammar structure.
5. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement according to claim 1, wherein the preset character length threshold is 5000 characters and can be dynamically adjusted according to the context window capacity of a large model.
6. The intelligent question-answering method based on the structured semantic indexing and the double-layer memory enhancement of claim 1, wherein the structured interaction log comprises a session_id field and a turn_ index, timestamp, interaction _data field, and the interaction_data field comprises an original query of a user, a post-rewrite query, an Agent thinking track, a tool name, a tool output abstract and a final response.
7. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement according to claim 1, wherein the triggering condition of asynchronous memory solidification is session end, system idle time exceeding a preset threshold or cross-day archiving, and the long-term information comprises user technical stack preference, core demand characteristics and historical interaction key facts.
8. The intelligent question-answering method based on the structured semantic indexing and the double-layer memory enhancement, which is disclosed in claim 1, is characterized in that the positive sample is an interaction record of user praise, the query after the rewrite, the search content and the final answer are extracted to construct a data set, the negative sample is an interaction record of user praise, and the interaction record is supplemented to the data set after the error type is manually marked.
9. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement as claimed in claim 1, wherein the search tool set is expanded through a tool registry, and the tool registry comprises tool names, function descriptions and parameter lists, so that custom configuration according to business scenes is supported.
10. The intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement according to claim 1, wherein the code skeleton extracts a reserved function signature, a parameter list and a main control flow structure, and concrete implementation details are replaced by annotation placeholders.

Description

Intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement Technical Field The invention belongs to the technical field of natural language processing and information retrieval, and relates to an intelligent question-answering method based on structural semantic indexing and double-layer memory enhancement. Background With the development of Large Language Models (LLMs), search enhancement generation (RAG) becomes the dominant paradigm of intelligent question-answering systems. However, existing RAG systems suffer from the following technical drawbacks: the unstructured data segmentation is unreasonable, namely the traditional fixed character/Token segmentation causes code semantic fracture and long text logic segmentation, so that the semantic features of different types of data cannot be adapted; The multi-mode processing capability is insufficient, the conversion efficiency of non-text files such as PDF, PPT and the like is low, and the semantic information of non-text elements such as tables, charts and the like is difficult to extract effectively; The retrieval accuracy is limited, namely, the recall rate of related information is low due to the modal gap between the user query and the code and the professional document, and the single retrieval algorithm cannot consider both semantic matching and keyword matching; The method has no long-term memory mechanism, namely, the context is lost after the session is ended, the user needs to repeatedly provide background information, and the interaction efficiency is low; the lack of self-optimizing closed loop relies on manually adjusting parameters and user feedback is not effectively utilized for system iterations. The comparison document 1 (CN 120337898B) focuses on a repeated scene of a punctuation, and relates to multi-mode information processing and semantic segmentation, but the problems of code retrieval, long-term memory, system self-evolution and the like are not solved, and the requirements of scenes such as technical research and development, code maintenance and the like cannot be met. Therefore, there is a need for an unstructured document processing scheme that combines differentiated segmentation, accurate retrieval, personalized interaction and self-evolution. Disclosure of Invention The invention aims to overcome the defects in the prior art and provides an intelligent question-answering method based on structural semantic indexing and double-layer memory enhancement. In order to achieve the purpose of the invention, the invention is implemented by adopting the following technical scheme. An intelligent question-answering method based on structured semantic indexing and double-layer memory enhancement comprises the following steps: (1) Preprocessing inquiry, namely rewriting the inquiry of a user, extracting key entities and setting retrieval filtering conditions; (2) The multi-tool dynamic routing based on the Agent comprises the steps of deploying a central routing Agent, analyzing the query intention of a user, and selecting a search tool from a preset search tool set according to semantic matching degree, wherein the search tool set comprises a document search tool, a code library search tool, a FAQ search tool and a networking search tool; (3) Structuring and semantically enhancing index building: the text data processing is to divide the file according to the title level of the file preferentially, and divide the file according to the maximum character length when no clear title or the text length exceeds a preset threshold value; Code data processing, namely converting a code file into an abstract syntax Tree AST by utilizing a Tree-sitter analyzer, taking the identified high-level semantic nodes as segmentation units, extracting code meta information, and calling a large language model to generate a function description abstract for each code segmentation block; the mixed index construction, namely calculating a search score by adopting a mixed search mechanism combining vector cosine similarity with a BM25 algorithm; (4) Setting a length threshold Tmax, returning the full text when the retrieval result length L is smaller than Tmax, extracting core information of a long text through a large language model when L is larger than or equal to Tmax, extracting a skeleton of a long code, and reserving a key logic structure; (5) Double-layer memory management, namely constructing a short-term scene buffer area for storing a structured interaction log, an asynchronous memory solidification module for regularly extracting long-term information and a long-term semantic knowledge base for realizing knowledge persistence storage and retrieval; (6) Recording complete links and user feedback of each interaction, generating positive and negative sample test data sets, verifying answer consistency through regression test when the system is updated, and realizing data-driven self-optimization. Furt