CN-122021963-A - Language processing model training method and language processing method

Abstract

The embodiment of the specification provides a language processing model training method and a language processing method, wherein the language processing model training method comprises the steps of determining a plurality of text processing tasks and initial text training data, carrying out data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, training an initial language processing model according to the task text training data corresponding to each text processing task to obtain a target language processing model, and carrying out initial text processing on target texts according to the initial text processing tasks in the plurality of text processing tasks under the condition that target texts are received by the target language processing model to obtain initial text processing results, carrying out target text processing on the initial text processing results according to the target text processing tasks in the plurality of text processing tasks to obtain target text processing results, and improving the accuracy of the target text processing results.

Inventors

• QIN YANG
• CHEN CHAO
• FU ZHIHANG
• CHEN ZE
• YE JIEPING

Assignees

• 阿里云计算有限公司

Dates

Publication Date

20260512

Application Date

20241112

Claims (20)

1. 1. A language processing model training method, comprising: Determining a plurality of text processing tasks and initial text training data; According to each text processing task in the plurality of text processing tasks, carrying out data processing on the initial text training data to obtain task text training data corresponding to each text processing task; training the initial language processing model according to the task text training data corresponding to each text processing task to obtain a target language processing model, The target language processing model is used for carrying out initial text processing on the target text according to initial text processing tasks in the text processing tasks under the condition that the target text is received, obtaining initial text processing results, and carrying out target text processing on the initial text processing results according to the target text processing tasks in the text processing tasks, so as to obtain target text processing results.
2. 2. The language processing model training method of claim 1, the plurality of text processing tasks comprising an initial text processing task and a target text processing task; The step of performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, includes: noise filtering is carried out on the initial text training data, and filtered text training data are obtained; performing initial text processing on the filtered text training data according to the initial text processing task to obtain initial task text training data corresponding to the initial text processing task; And performing target text processing on the filtered text training data according to the target text processing task to obtain target task text training data corresponding to the target text processing task.
3. 3. The language processing model training method according to claim 2, wherein training the initial language processing model according to the task text training data corresponding to each text processing task to obtain the target language processing model comprises: Training the initial language processing model according to the initial task text training data corresponding to the initial text processing task and the target task text training data corresponding to the target text processing task to obtain the target language processing model.
4. 4. The language processing model training method according to claim 1 or 2, wherein the plurality of text processing tasks includes an initial text processing task including a text generation task and a data filtering task, The step of performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, includes: Extracting example information of a data source sample in filtered text training data according to the text generation task to obtain the example information of the data source sample, wherein the filtered text training data is obtained by carrying out noise filtering on the initial text training data, and the filtered text training data comprises the initial text sample, a target language data sample corresponding to the initial text sample and the data source sample; Obtaining first target text training data corresponding to the text generation task according to the initial text sample, a target language data sample corresponding to the initial text sample and the example information; According to the data screening task, extracting data source sample information from the target language data samples in the filtered text training data to obtain associated data source sample information corresponding to the initial text samples; And obtaining second target text training data corresponding to the data screening task according to the initial text sample, the associated data source sample information and the example information.
5. 5. The language processing model training method according to claim 4, wherein training the initial language processing model according to the task text training data corresponding to each text processing task to obtain the target language processing model comprises: Determining the initial text sample and the example information in the first target text training data as a first prompt text, and determining the target language data sample in the first target text training data as a first reply tag; Determining the initial text sample, the example information and the target request in the second target text training data as a second prompt text, and determining the associated data source sample information in the second target text training data as a second reply tag; Training the initial language processing model according to the first prompt text, the first reply label, the second prompt text and the second reply label to obtain the target language processing model.
6. 6. The language processing model training method of claim 4, the plurality of text processing tasks further comprising a target text processing task, the target text processing task comprising a noise correction task; The step of performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, and the step of further comprising: according to the noise correction task, carrying out data replacement on the target language data sample in the filtered text training data to obtain a replaced language data sample; and obtaining third target text training data corresponding to the noise correction task according to the initial text sample, the alternative language data sample, the target language data sample and the example information.
7. 7. The language processing model training method according to claim 6, wherein training the initial language processing model according to the task text training data corresponding to each text processing task to obtain the target language processing model comprises: Determining the initial text sample and the example information in the first target text training data as a first prompt text, and determining the target language data sample in the first target text training data as a first reply tag; Determining the initial text sample, the example information and the target request in the second target text training data as a second prompt text, and determining the associated data source sample information in the second target text training data as a second reply tag; Determining the initial text sample, the alternative language data sample and the example information in the third target text training data as a third prompt text, and determining the target language data sample in the third target text training data as a third reply tag; Training the initial language processing model according to the first prompt text, the first reply label, the second prompt text, the second reply label, the third prompt text and the third reply label to obtain the target language processing model.
8. 8. The language processing model training method of claim 4, the plurality of text processing tasks further comprising a target text processing task, the target text processing task comprising a data writing task; The step of performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, and the step of further comprising: according to the data writing task, carrying out data truncation on the target language data sample in the filtered text training data to obtain truncated language data; And obtaining fourth target text training data corresponding to the data writing task according to the initial text sample, the truncated language data, the target language data sample and the example information.
9. 9. The language processing model training method according to claim 8, wherein training the initial language processing model according to the task text training data corresponding to each text processing task to obtain the target language processing model comprises: Determining the initial text sample and the example information in the first target text training data as a first prompt text, and determining the target language data sample in the first target text training data as a first reply tag; Determining the initial text sample, the example information and the target request in the second target text training data as a second prompt text, and determining the associated data source sample information in the second target text training data as a second reply tag; determining the initial text sample, the truncated language data and the example information in the fourth target text training data as a fourth prompt text, and determining the target language data sample in the fourth target text training data as a fourth reply tag; Training the initial language processing model according to the first prompt text, the first reply label, the second prompt text, the second reply label, the fourth prompt text and the fourth reply label to obtain the target language processing model.
10. 10. The language processing model training method of claim 4, the plurality of text processing tasks further comprising a target text processing task, the target text processing task comprising a noise correction task and a data writing task; The step of performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, and the step of further comprising: according to the noise correction task, carrying out data replacement on the target language data sample in the filtered text training data to obtain a replaced language data sample; obtaining third target text training data corresponding to the noise correction task according to the initial text sample, the alternative language data sample, the target language data sample and the example information; according to the data writing task, carrying out data truncation on the target language data sample in the filtered text training data to obtain truncated language data; And obtaining fourth target text training data corresponding to the data writing task according to the initial text sample, the truncated language data, the target language data sample and the example information.
11. 11. The language processing model training method according to claim 10, wherein training the initial language processing model according to the task text training data corresponding to each text processing task to obtain the target language processing model comprises: And training the initial language processing model according to the first target text training data corresponding to the text generation task, the second target text training data corresponding to the data screening task, the third target text training data corresponding to the noise correction task and the fourth target text training data corresponding to the data writing task to obtain the target language processing model.
12. 12. The language processing model training method according to claim 1 or 2, the plurality of text processing tasks including a target text processing task including a noise correction task and a data writing task; The step of performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task, includes: Extracting example information of a data source sample in filtered text training data to obtain example information of the data source sample, wherein the filtered text training data is obtained by noise filtering of the initial text training data, and the filtered text training data comprises the initial text sample, a target language data sample corresponding to the initial text sample and the data source sample; according to the noise correction task, carrying out data replacement on the target language data sample in the filtered text training data to obtain a replaced language data sample; obtaining third target text training data corresponding to the noise correction task according to the initial text sample, the alternative language data sample, the target language data sample and the example information; according to the data writing task, carrying out data truncation on the target language data sample in the filtered text training data to obtain truncated language data; And obtaining fourth target text training data corresponding to the data writing task according to the initial text sample, the truncated language data, the target language data sample and the example information.
13. 13. The language processing model training method according to claim 12, wherein training the initial language processing model according to the task text training data corresponding to each text processing task to obtain the target language processing model comprises: and training the initial language processing model according to the third target text training data corresponding to the noise correction task and the fourth target text training data corresponding to the data writing task to obtain the target language processing model.
14. 14. The language processing model training method according to claim 1, wherein the performing data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task includes: Noise filtering is carried out on initial text training data by utilizing a discriminator to obtain filtered text training data, wherein the initial text training data comprises initial text samples, target language data samples corresponding to the initial text samples and data source samples; and carrying out data processing on the filtered text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task.
15. 15. The language processing model training method of claim 14, the noise filtering the initial text training data with a discriminator to obtain filtered text training data, comprising: Inputting the initial text training data into the discriminator to obtain a classification result of the initial text training data, wherein the discriminator is obtained through the training of the marked text training data of the positive sample and the negative sample; And determining the initial text training data with the classification result being a positive sample as the filtered text training data.
16. 16. A language processing method, comprising: Determining a target text; According to an initial text processing task of a target language processing model, performing initial text processing on the target text to obtain an initial text processing result; And performing target text processing on the initial text processing result according to the target text processing task of the target language processing model to obtain a target text processing result.
17. 17. The language processing method of claim 16, wherein the initial text processing tasks include a text generation task and a data screening task; the initial text processing task according to the target language processing model performs initial text processing on the target text to obtain an initial text processing result, including: Extracting associated data source information of the target text from a target data source corresponding to the target language processing model according to the data screening task of the target language processing model; And predicting the target text based on the associated data source information according to the text generation task of the target language processing model to obtain the initial text processing result.
18. 18. The language processing method of claim 16, the target text processing task comprising a noise correction task; The target text processing task according to the target language processing model performs target text processing on the initial text processing result to obtain a target text processing result, including: according to the noise correction task of the target language processing model, judging the correctness of the initial text processing result to obtain a judging result; And determining the initial text processing result as the target text processing result when the judging result is correct, And correcting the initial text processing result according to the noise correction task under the condition that the judging result is wrong, obtaining a corrected text processing result, and determining the target text processing result according to the corrected text processing result.
19. 19. The language processing method of claim 16, wherein the target text processing task comprises a data writing task; The target text processing task according to the target language processing model performs target text processing on the initial text processing result to obtain a target text processing result, including: judging whether the initial text processing result meets an initial data writing condition according to the data writing task of the target language processing model; in the case that the initial text processing result does not satisfy the initial data writing condition, determining the initial text processing result as the target text processing result, And under the condition that the initial text processing result meets the initial data writing condition, writing the initial text processing result according to the data writing task to obtain the target text processing result.
20. 20. The language processing method of claim 18, wherein the target text processing task further comprises a data writing task; the determining the text processing result according to the corrected text processing result comprises the following steps: judging whether the corrected text processing result meets the target data writing condition according to the data writing task of the target language processing model, In the case where the corrected text processing result does not satisfy the target data writing condition, determining the corrected text processing result as the target text processing result, And under the condition that the corrected text processing result meets the target data writing condition, writing the corrected text processing result according to the data writing task to obtain the target text processing result.

Description

Language processing model training method and language processing method Technical Field The embodiment of the specification relates to the technical field of computers, in particular to a language processing model training method and a language processing method. Background The realization of Text-to-SQL generation (Text 2SQL, structured Query Language, structured query language) by using a large language model (LLMs, large Language Models) is a current technical scheme based on massive database questions and answers, and has great commercial value. The existing method is mostly dependent on a general large language model for carrying out context learning to realize text-to-SQL generation, but the method can cause a large amount of reasoning overhead in deployment in a practical scene. Meanwhile, when migrating to a small-size LLM, the effectiveness of text-to-SQL generation may be reduced, showing lower versatility, and limiting practical applications. In addition, existing supervised fine Tuning (SFT, supervised Fine-Tuning) methods for Text2SQL focus mostly on a single task, fine Tuning of a single model, with the risk of overfitting. Disclosure of Invention In view of this, the embodiments of the present disclosure provide a language processing model training method and a language processing method. One or more embodiments of the present disclosure relate to a language processing model training apparatus, a language processing apparatus, a computing device, a computer readable storage medium, and a computer program product, so as to solve the technical defect that when a large language model is used to generate text to SQL in the prior art, the large language model can generate a lot of reasoning overhead, and when a single task is used to perform supervised fine tuning on the large language model, the generated SQL is inaccurate. According to a first aspect of embodiments of the present specification, there is provided a language processing model training method, including: Determining a plurality of text processing tasks and initial text training data; According to each text processing task in the plurality of text processing tasks, carrying out data processing on the initial text training data to obtain task text training data corresponding to each text processing task; training the initial language processing model according to the task text training data corresponding to each text processing task to obtain a target language processing model, The target language processing model is used for carrying out initial text processing on the target text according to initial text processing tasks in the text processing tasks under the condition that the target text is received, obtaining initial text processing results, and carrying out target text processing on the initial text processing results according to the target text processing tasks in the text processing tasks, so as to obtain target text processing results. According to a second aspect of embodiments of the present specification, there is provided a language processing model training apparatus, comprising: A determining module configured to determine a plurality of text processing tasks, and initial text training data; the obtaining module is configured to perform data processing on the initial text training data according to each text processing task in the plurality of text processing tasks to obtain task text training data corresponding to each text processing task; The training module is configured to train the initial language processing model according to task text training data corresponding to each text processing task to obtain a target language processing model, wherein the target language processing model is used for carrying out initial text processing on the target text according to the initial text processing tasks in the plurality of text processing tasks under the condition that the target text is received to obtain an initial text processing result, and carrying out target text processing on the initial text processing result according to the target text processing tasks in the plurality of text processing tasks to obtain a target text processing result. According to a third aspect of embodiments of the present specification, there is provided a language processing method, including: Determining a target text; According to an initial text processing task of a target language processing model, performing initial text processing on the target text to obtain an initial text processing result; And performing target text processing on the initial text processing result according to the target text processing task of the target language processing model to obtain a target text processing result. According to a fourth aspect of embodiments of the present specification, there is provided a language processing apparatus comprising: A text determination module configured to determine a target text; the initial obtaining module is configur