CN-122021664-A - Novel federal knowledge learning model-based distributed intelligent question-answering method

CN122021664ACN 122021664 ACN122021664 ACN 122021664ACN-122021664-A

Abstract

The invention relates to a distributed intelligent question-answering method based on a novel federal knowledge learning model, which belongs to the field of natural language processing, and comprises the steps that each client side utilizes local privacy data to finely tune a private downstream task large model, and a server utilizes a public data set to finely tune a global model so as to enhance knowledge understanding capability; the method comprises the steps of enabling a central server to decompose task flows into required demands and send the demands to a client, enabling the client to generate corresponding knowledge fragments according to the received demands and upload the knowledge fragments to the server, enabling the server to aggregate fragmented knowledge received from the client to form complete task auxiliary knowledge, and enabling a server large model to combine the task auxiliary knowledge to generate task responses. The invention introduces the task decomposition, knowledge aggregation and de-identification module, not only ensures high performance, but also provides strong privacy protection.

Inventors

WANG FANGXIN
SUN WEI
WANG XIANDA
LIANG ZHICHENG
GONG TIANYI
CUI SHUGUANG

Assignees

香港中文大学（深圳）
深圳市未来智联网络研究院

Dates

Publication Date: 20260512
Application Date: 20260414

Claims (9)

1. A novel federal knowledge learning model-based distributed intelligent question-answering method is characterized by comprising the following steps: S1, each client side utilizes local privacy data to finely tune a private downstream task large model, and meanwhile a server utilizes a public data set to finely tune a global model so as to enhance knowledge understanding capability; S2, the central server decomposes the task flow into required demands and issues the demands to the client; s3, the client generates a corresponding knowledge fragment according to the received requirement and uploads the knowledge fragment to the server; S4, the server aggregates the fragmented knowledge received from the client to form complete task auxiliary knowledge; S5, the server large model combines task auxiliary knowledge to generate task responses.
2. The distributed intelligent question-answering method based on the novel federal knowledge learning model of claim 1, wherein the S1 specifically comprises the following contents: S101, model selection, namely, the client terminal individuates and selects a base large model according to a private downstream task, and model fine tuning, namely, the client terminal utilizes a held privacy data set Fine tuning a base large model to maintain a downstream task large model ; S102, constructing a general knowledge understanding large model by the server, wherein the server uses a public data set Fine-tuning local large models to develop a global model that performs well in knowledge reasoning and logic understanding 。
3. The distributed intelligent question-answering method based on the novel federal knowledge learning model of claim 1, wherein the S2 specifically comprises the following contents: Demand task flow for server large models The task requirements of the server are dynamically added, removed or modified, and the server firstly utilizes a requirement decomposition module Each original query of task stream Resolution into domain-specific knowledge requirements At the same time, the server uses privacy to de-identify the module The privacy-sensitive entry is removed and the server then distributes the requirements to each client.
4. The distributed intelligent question-answering method based on novel federal knowledge learning model as set forth in claim 3, wherein said demand decomposition module For a summary and feature extraction language model T5, the server uses the feature extraction function of T5 to decompose the task flow prior to demand allocation, and the specific decomposition process includes: the server constructs a prompt word containing a task disassembly instruction in advance; performing text splicing on the original intelligent question-answering task watershed prompt words, constructing a text-to-text input format required by a T5 model, and inputting the text-to-text input format into the T5 model; Extracting core semantic features in a task stream by utilizing natural language understanding and generating capability of a T5 model, and directly generating a structured output text containing a plurality of subtasks; and the server performs character segmentation on the structured output text according to the preset separation identifier, so that a plurality of independent domain-specific knowledge requirements are obtained and distributed to the client.
5. The distributed intelligent question-answering method based on the novel federal knowledge learning model of claim 1, wherein the step S3 specifically comprises the following steps: Each client Generating a client knowledge set based on received requirements Comprising a plurality of domain-specific knowledge segments, each client requiring a de-identification module Ensuring that the generated domain knowledge has no sensitive information, wherein n is the number of elements in the client knowledge set; Marking module The specific implementation mechanism comprises a de-identification module The method comprises the steps that personalized privacy de-identification prompts are configured in advance, a client-side performs text splicing on received knowledge demands and hard prompts to form a context instruction with privacy constraint, then the client-side calls a local fine-tuned downstream task big model, constraint text generation is performed based on the context instruction, and therefore desensitized knowledge segments which do not contain sensitive entities are directly output at the source.
6. The distributed intelligent question-answering method based on the novel federal knowledge learning model of claim 1, wherein the step S4 specifically comprises the following steps: After knowledge generation is completed, the clients upload the de-identified relevant knowledge segments to a server, and the server obtains all knowledge sets which are contributed by the domain model of each client and relevant to the task Server utilizing demand decomposing module To aggregate knowledge, clean up multiple knowledge segments and integrate into a high-quality task auxiliary knowledge item according to the original query Selecting related knowledge items and integrating the related knowledge items into unified knowledge auxiliary items Where N is the number of elements in the overall knowledge set.
7. The distributed intelligent question-answering method based on the novel federal knowledge learning model of claim 6, wherein the step S5 specifically comprises the following steps: the server big model uses its knowledge reasoning and logic understanding capabilities to query in original And knowledge assistance entries Deriving a final task response ; For task query collections Each original query of the task flow in The server large model generates corresponding task response Form a response set 。
8. The distributed intelligent question-answering method based on the novel federal knowledge learning model according to any one of claims 1-7, wherein the method further comprises: And S6, maintaining the obtained triplet data set by the server, performing instruction fine adjustment on the global model by utilizing the data set, and integrating the knowledge of each client.
9. The distributed intelligent question-answering method based on the novel federal knowledge learning model of claim 8, wherein the step S6 specifically comprises the following steps: with the increase of communication rounds, the server will accumulate more and more comprehensive data sets of historical task driving records The system consists of knowledge, tasks and response triples, and for the triples, a server updates downstream knowledge across fields by using instruction trimming to further enhance the knowledge reasoning capability.

Description

Novel federal knowledge learning model-based distributed intelligent question-answering method Technical Field The invention relates to the field of natural language processing, in particular to a novel federal knowledge learning model-based distributed intelligent question-answering method. Background Large-scale language models (Large Language Models, LLMs) have achieved significant success in natural language processing (Natural Language Processing, NLP) tasks in recent years, and are the standard for processing a variety of complex tasks in this field. However, as these models grow in size, several new challenges are presented, despite the increasingly significant performance advantages they can achieve. Among the most significant problems are the need for efficient training mechanisms and the need for large amounts of data. To address these challenges, the use of federal learning (FEDERATED LEARNING, FL) in LLMs is of great interest. The FL paradigm significantly reduces the burden of training a centralized server by distributing training tasks across multiple client devices, utilizing the decentralized data of each client device vendor. Compared with the traditional centralized method, the FL can effectively share the calculation pressure and improve the data privacy. However, in practice, the large amount of parameters in a large-scale language model limits the straightforward efficiency of FL because excessive computational and communication costs are required to aggregate and broadcast the complete model during each round of federal learning. In a specific distributed intelligent question-answering scenario (such as medical record question-answering of a cross-medical institution and wind control text analysis of a cross-financial institution), a high-quality text data set usually belongs to highly sensitive privacy data, and if a traditional federal learning framework is directly applied to a distributed question-answering system, the risk of text privacy disclosure caused by model reverse engineering still exists. Therefore, an improved architecture for intelligent question-answering scenarios is needed, and huge model parameter exchange is converted into lightweight desensitized knowledge segment exchange, so that the legal compliance and data island problems faced by a large model in the grounding of specific applications are solved. In order to alleviate this problem, the prior art introduces a parameter efficient fine tuning technique into the FL framework, for example, the ground rank adaptation (LoRA) method improves the overall efficiency of computation and communication in the FL process by minimizing the number of parameters that need fine tuning, and successfully preserves most of the performance of the model without fully adjusting all the model parameters. Despite extensive research attempting to integrate FL with LLMs, this mode currently faces inherent limitations. The direct application of the traditional federal learning framework to LLMs presents a number of challenges because the traditional FL framework does not fully meet the specific needs of LLMs, thereby increasing system complexity. Conventional FL requires alignment of multiple language models, checkpoints, and alignments of word segmentors to achieve efficient model aggregation. This requirement significantly limits utility and scalability, unless techniques such as distillation are introduced by knowledge. Although research explores the potential of FL to promote LLMs multi-domain knowledge fusion, the resulting aggregate model tends to be less specialized than the original model in domain-specific expertise, resulting in impaired expertise. The conventional FL must handle various specialized LLMs parameters while integrating heterogeneous data knowledge in multiple fields. Balancing the need for synchronization and heterogeneity not only increases the time requirements, but also exacerbates the communication and computational complexity. In the LLMs context, the privacy protection mechanism of FL is particularly vulnerable, bringing a huge risk of privacy leakage. Especially when the server loads the client's hinted parameters, it may cause the context-sensitive model to obtain private data. Disclosure of Invention The invention aims to overcome the defects of the prior art, provides a distributed intelligent question-answering method based on a novel federal knowledge learning model, and solves the defects in the prior art. The invention aims at realizing the distributed intelligent question-answering method based on the novel federal knowledge learning model, which comprises the following steps: S1, each client side utilizes local privacy data to finely tune a private downstream task large model, and meanwhile a server utilizes a public data set to finely tune a global model so as to enhance knowledge understanding capability; S2, the central server decomposes the task flow into required demands and issues the demand