CN-119441453-B - Question-answering task processing method, device, equipment, storage medium and program product

CN119441453BCN 119441453 BCN119441453 BCN 119441453BCN-119441453-B

Abstract

The invention provides a question-answering task processing method, a question-answering task processing device, question-answering task processing equipment, question-answering task processing storage medium and question-answering task processing program product, and is applied to the technical field of natural language processing. The method comprises the steps of obtaining input questions and question and answer examples of a question and answer task, determining a target example with highest similarity with the input questions from the question and answer examples, determining a plurality of LORA modules with highest similarity with the target example from a pre-trained low-rank adaptive LORA module library, carrying out fusion processing on the plurality of LORA modules and a first question and answer model by adopting a non-gradient parameter optimization technology to obtain a second question and answer model, and inputting the input questions into the second question and answer model to obtain corresponding answers.

Inventors

HE SHIZHU
LIU KANG
ZHAO JUN
WANG ZHIQI

Assignees

中国科学院自动化研究所

Dates

Publication Date: 20260505
Application Date: 20240913

Claims (6)

1. The question-answering task processing method is characterized by comprising the following steps of: Acquiring an input problem and a question and answer example of a question and answer task; determining a target instance with highest similarity to the input problem from the question-answer instances, and determining a plurality of LORA modules with highest similarity to the target instance from a pre-trained low-rank adaptive LORA module library; Carrying out fusion processing on the LORA modules and the first question-answer model by adopting a non-gradient parameter optimization technology to obtain a second question-answer model, and inputting the input questions into the second question-answer model to obtain corresponding answers; the determining the target instance with the highest similarity with the input problem from the question-answer instances comprises the following steps: based on a sentence vectorization model, carrying out feature extraction on the input and output of the question-answering instance to obtain an instance feature vector of the question-answering instance, and carrying out feature extraction on the input problem to obtain an input feature vector; the cosine similarity of the input feature vector and each instance feature vector is calculated, and a question-answer instance corresponding to the instance feature vector with the highest similarity is determined as the target instance; Wherein the determining, from the pre-trained low-rank adaptive LORA module library, a plurality of LORA modules with highest similarity to the target instance comprises: Based on a sentence vectorization model, vectorizing each piece of training data in a training data set of a LORA module, and determining a vector average value of the training data set as a representative vector of the LORA module; calculating cosine similarity between the instance feature vector of the target instance and the representative vector of each LORA module; And sequencing the LORA modules in the LORA module library according to the sequence of cosine similarity from high to low, and determining the plurality of LORA modules with highest similarity.
2. The method for processing a question-answering task according to claim 1, wherein the fusing the plurality of LORA modules with the first question-answering model by using a non-gradient parameter optimization technique to obtain a second question-answering model includes: Adjusting fusion parameters by adopting a non-gradient parameter optimization technology, and weighting and adding the plurality of LORA modules according to the fusion parameters to obtain a LORA module group of the target instance; And carrying out fusion processing on the LORA module group and the first question-answering model to obtain the second question-answering model.
3. The question-answering task processing device is characterized by comprising an acquisition module and a processing module; the acquisition module is used for acquiring the input questions and the question and answer examples of the question and answer task; The processing module is used for determining a target instance with highest similarity to the input question from the question-answering instances, determining a plurality of LORA modules with highest similarity to the target instance from a pre-trained low-rank adaptive LORA module library, carrying out fusion processing on the plurality of LORA modules and a first question-answering model by adopting a non-gradient parameter optimization technology to obtain a second question-answering model, and inputting the input question into the second question-answering model to obtain a corresponding answer; the determining the target instance with the highest similarity with the input problem from the question-answer instances comprises the following steps: based on a sentence vectorization model, carrying out feature extraction on the input and output of the question-answering instance to obtain an instance feature vector of the question-answering instance, and carrying out feature extraction on the input problem to obtain an input feature vector; the cosine similarity of the input feature vector and each instance feature vector is calculated, and a question-answer instance corresponding to the instance feature vector with the highest similarity is determined as the target instance; Wherein the determining, from the pre-trained low-rank adaptive LORA module library, a plurality of LORA modules with highest similarity to the target instance comprises: Based on a sentence vectorization model, vectorizing each piece of training data in a training data set of a LORA module, and determining a vector average value of the training data set as a representative vector of the LORA module; calculating cosine similarity between the instance feature vector of the target instance and the representative vector of each LORA module; And sequencing the LORA modules in the LORA module library according to the sequence of cosine similarity from high to low, and determining the plurality of LORA modules with highest similarity.
4. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the question-answering task processing method according to any one of claims 1 to 2 when the computer program is executed by the processor.
5. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a question-answering task processing method according to any one of claims 1 to 2.
6. A computer program product comprising a computer program which, when executed by a processor, implements a method of question-answering task processing according to any one of claims 1 to 2.

Description

Question-answering task processing method, device, equipment, storage medium and program product Technical Field The present invention relates to the field of natural language processing technologies, and in particular, to a method, an apparatus, a device, a storage medium, and a program product for processing a question-answering task. Background In the field of natural language processing, large language models are often not good at handling unseen tasks, i.e. completely new tasks beyond the task of pre-training of the large language model. In the prior art, in order to solve the task generalization problem of a large language model, the most direct method is to continuously pretrain the model. However, direct training of the model requires a large amount of training data, and also consumes more computing resources due to the influence of gradient back-propagation. Disclosure of Invention The invention provides a question-answering task processing method apparatus, device, storage medium, and program product, the method is used for solving the problems that a large amount of training data is needed for directly training the model in the prior art, and more computing resources are consumed. The invention provides a question and answer task processing method which comprises the steps of obtaining an input question and a question and answer example of a question and answer task, determining a target example with highest similarity to the input question from the question and answer example, determining a plurality of LORA modules with highest similarity to the target example from a pre-trained low-rank adaptive LORA module library, carrying out fusion processing on the plurality of LORA modules and a first question and answer model by adopting a non-gradient parameter optimization technology to obtain a second question and answer model, and inputting the input question into the second question and answer model to obtain a corresponding answer. The method for processing the question-answering task comprises the steps of carrying out feature extraction on input and output of a question-answering example based on a sentence vectorization model to obtain an example feature vector of the question-answering example, carrying out feature extraction on the input question to obtain an input feature vector, calculating cosine similarity of the input feature vector and each example feature vector, and determining the question-answering example corresponding to the example feature vector with the highest similarity as the target example. The method for processing the question-answering task comprises the steps of determining a plurality of LORA modules with highest similarity to a target instance from a pre-trained low-rank adaptive LORA module library, vectorizing each piece of training data in training data sets of the LORA modules based on a sentence vectorization model, determining vector average values of the training data sets as representative vectors of the LORA modules, calculating cosine similarity between instance feature vectors of the target instance and representative vectors of each LORA module, sequencing the LORA modules in the LORA module library according to the sequence of the cosine similarity from high to low, and determining the plurality of LORA modules with highest similarity. The method for processing the question-answering task comprises the steps of adopting a non-gradient parameter optimization technology to conduct fusion processing on a plurality of LORA modules and a first question-answering model to obtain a second question-answering model, adjusting fusion parameters by adopting the non-gradient parameter optimization technology, adding the plurality of LORA modules according to the fusion parameters in a weighted mode to obtain a LORA module group of the target instance, and conducting fusion processing on the LORA module group and the first question-answering model to obtain the second question-answering model. The invention further provides a question and answer task processing device which comprises an acquisition module and a processing module, wherein the acquisition module is used for acquiring input questions and question and answer examples of the question and answer tasks, the processing module is used for determining a target example with highest similarity with the input questions from the question and answer examples, determining a plurality of LORA modules with highest similarity with the target example from a pre-trained low-rank adaptive LORA module library, and carrying out fusion processing on the plurality of LORA modules and a first question and answer model by adopting a non-gradient parameter optimization technology to obtain a second question and answer model, and inputting the input questions into the second question and answer model to obtain corresponding answers. The question-answering task processing device is used for extracting the characteristics of the inp