US-12619604-B2 - Prompt tuning using one or more machine-learned models

Abstract

Systems and methods for prompt tuning can leverage semantic searching for determining similar prompts to use for retraining. A prompt can be generated then searched to find the similar prompts. Data related to the similar prompts can then be utilized for prompt tuning. Moreover, systems and methods for prompt tuning can generate and utilize a meta-prompt to reduce the computational cost of generating prompts. The prompt tuning techniques can be implemented as part of a prompt tuning application programming interface (API).

Inventors

• Brian David Lester
• Rami Eid Sammour Al-Rfou
• Noah JG Constant

Assignees

• GOOGLE LLC

Dates

Publication Date

20260505

Application Date

20210820

Claims (17)

1. 1 . A computer-implemented method, the method comprising: obtaining, by a computing system comprising one or more processors, an example dataset from a user, wherein the example dataset comprises a plurality of training examples and respective labels; processing, by the computing system, the example dataset with a machine-learned model to train a first prompt, wherein the first prompt is trained to generate data associated with the example dataset, wherein the first prompt comprises one or more learned parameters representative of a first task; determining, by the computing system, one or more second prompts are associated with the first prompt, wherein the one or more second prompts comprise one or more pretrained prompts; generating, by the computing system, a similarity score for each of the one or more second prompts, wherein the similarity scores are determined based on a similarity between the first prompt and the second prompt; and generating, by the computing system, an augmented first prompt with the machine-learned model based at least in part on performing multi-task learning based on the one or more second prompts, wherein generating the augmented first prompt comprises at least one of training or fine-tuning the first prompt on one or more second prompt datasets associated with the one or more second prompts with a highest similarity score.
2. 2 . The method of claim 1 , wherein determining the one or more second prompts comprises: performing, by the computing system, a search of a prompt repository to identify similar prompts to the first prompt; and determining, by the computing system, the one or more second prompts based on the search of the prompt repository.
3. 3 . The method of claim 1 , wherein generating the augmented first prompt comprises: obtaining, by the computing system, second prompt metadata associated with the one or more second prompts; and processing, by the computing system, the second prompt metadata and the example dataset with the machine-learned model to generate the augmented first prompt.
4. 4 . The method of claim 3 , wherein the second prompt metadata comprises one or more second prompt datasets associated with the one or more second prompts.
5. 5 . The method of claim 1 , wherein generating the augmented first prompt comprises multi-task learning based on the one or more second prompts.
6. 6 . The method of claim 1 , wherein generating the augmented first prompt comprises curriculum learning based at least in part on the one or more second prompts.
7. 7 . The method of claim 1 , wherein the one or more second prompts are obtained from a plurality of stored prompts stored on a server computing system.
8. 8 . The method of claim 1 , further comprising: obtaining, by the computing system, a prompt tuning training application programming interface; and wherein the machine-learned model comprises a prompt tuning model of the prompt tuning training application programming interface.
9. 9 . The method of claim 1 , wherein determining the one or more second prompts comprises a semantic search comprising at least one of an L2 metric, a cosine metric, or a max product metric.
10. 10 . The method of claim 1 , wherein the one or more second prompts are determined based on a semantic search of a library of prompts stored with associated metadata for each respective prompt of the library of prompts.
11. 11 . The method of claim 10 , wherein the associated metadata comprises at least one of a frozen model utilized for training a respective second prompt, a training date for the respective second prompt, or a training dataset for the respective second prompt.
12. 12 . A computing system, the system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising: obtaining a plurality of training datasets; processing the plurality of training datasets to train a training prompt for each respective dataset, wherein a prompt comprises one or more learned parameters representative of a task; selecting one or more canonical examples from each of the plurality of training datasets to determine a plurality of canonical examples; generating an example dataset based on the plurality of canonical examples; generating a meta-prompt based on the example dataset, wherein the meta-prompt is configured to generate a predicted prompt based on the plurality of canonical examples; obtaining an aggregated dataset, wherein the aggregated dataset comprises a task description; processing the task description and the meta-prompt with prompt creation model to generate a task-specific prompt for the task description; processing the task-specific prompt and the task description with a prediction model to generate a prediction; evaluating a loss function that evaluates the prediction; and adjusting one or more parameters of the meta-prompt based at least in part on the loss function.
13. 13 . The computing system of claim 12 , wherein the operations further comprise: wherein the aggregated dataset comprises an aggregated example and a respective label; processing the aggregated example and the meta-prompt with the prompt creation model to generate the task-specific prompt for the aggregated example; processing the task-specific prompt and the aggregated example with the prediction model to generate a second prediction; evaluating a second loss function that evaluates a difference between the prediction and the respective label; and adjusting the one or more parameters of the meta-prompt based at least in part on the second loss function.
14. 14 . The computing system of claim 12 , wherein the plurality of training datasets are obtained from a user computing device, and wherein the meta-prompt comprises a user-specific meta-prompt.
15. 15 . The computing system of claim 12 , wherein selecting the one or more canonical examples from each of the plurality of training datasets comprises determining a specific training dataset from each of the plurality of training datasets based on a dataset-specific prompt generated based at least in part on the specific training dataset and a frozen model.
16. 16 . The computing system of claim 12 , wherein the operations further comprise: storing the meta-prompt on a server computing system; receiving a prompt request from a user computing device; generating a requested prompt based on the prompt request and the meta-prompt; and sending the requested prompt to the user computing device.
17. 17 . One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a server computing system to perform operations, the operations comprising: obtaining, via an application programming interface, a training example and a prompt from a user computing device, wherein the training example are associated with the prompt, and wherein the prompt comprises a set of learned parameters that represent a task; processing the training example and the prompt with a frozen model to generate one or more prompt gradients, wherein the prompt gradient is based at least in part on a prediction generated based on the training example, wherein parameters of the frozen model are held fixed during the training of the prompt, and wherein processing the training example and the prompt with the frozen model to generate the one or more prompt gradients comprises: a forward pass of the frozen model with the training example and the prompt; and a backwards pass to return the prompt gradients; sending, via the application programming interface, the one or more prompt gradients to the user computing device for generating an augmented prompt.

Description

RELATED APPLICATIONS This application claims priority under 35 U.S.C. § 371 to and the benefit of International Patent Application No. PCT/US2021/046849 filed on Aug. 20, 2021. International Patent Application No. PCT/US2021/046849 is hereby incorporated by reference in its entirety. FIELD The present disclosure relates generally to prompt tuning. More particularly, the present disclosure relates to generation and/or tuning of prompts with semantic searches, meta-prompts trained to generate prompts, and visual prompt tuning. BACKGROUND Data collection can be hard for users as they start with just a few labeled examples for evaluation and training. In some current systems, the expansion of the data for prompt training can be based on similarity between examples, without consideration of the context of the task itself. The lack of consideration can cause the selection of examples that are not pertinent to a desired task. Selecting appropriate datasets to augment, or tune, a prompt can be difficult based on the lack of search refinement for the particular task. Additionally, some current systems can include transfer learning. Transfer learning for large pretrained models can show great results and can be state of the art on a plurality of tasks. However, transfer learning can be difficult for many people to use due to computational resources needed and parallel computing expertise. SUMMARY Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments. One example aspect of the present disclosure is directed to a computer-implemented method. The method can include obtaining, by a computing system including one or more processors, an example dataset from a user. The method can include processing, by the computing system, the example dataset with a machine-learned model to train a first prompt. In some implementations, the first prompt can be trained to generate data associated with the example dataset. The method can include determining, by the computing system, one or more second prompts are associated with the first prompt and generating, by the computing system, an augmented first prompt with the machine-learned model based at least in part on the one or more second prompts. In some implementations, determining the one or more second prompts can include: performing, by the computing system, a search of a prompt repository to identify similar prompts to the first prompt and determining, by the computing system, the one or more second prompts based on the search of the prompt repository. The method can include generating, by the computing system, a similarity score for each of the one or more second prompts. In some implementations, the similarity scores can be determined based on a similarity between the first prompt and the second prompt. In some implementations, generating the augmented first prompt can include retraining the first prompt on one or more second prompt datasets associated with the one or more second prompts with a highest similarity score. Generating the augmented first prompt can include obtaining, by the computing system, second prompt metadata associated with the one or more second prompts and processing, by the computing system, the second prompt metadata and the example dataset with the machine-learned model to generate the augmented first prompt. In some implementations, the second prompt metadata can include one or more second prompt datasets associated with the one or more second prompts. Generating the augmented first prompt can include multi-task learning based on the one or more second prompts. In some implementations, generating the augmented first prompt can include curriculum learning based at least in part on the one or more second prompts. The one or more second prompts can be obtained from a plurality of stored prompts stored on a server computing system. The method can include obtaining, by the computing system, a prompt tuning training application programming interface, and the machine-learned model can include a prompt tuning model of the prompt tuning training application programming interface. In some implementations, determining the one or more second prompts can include a semantic search including at least one of an L2 metric, a cosine metric, or a max product metric. The one or more second prompts can be determined based on a semantic search of a library of prompts stored with associated metadata for each respective prompt of the library of prompts. The associated metadata can include at least one of a frozen model utilized for training a respective second prompt, a training date for the respective second prompt, or a training dataset for the respective second prompt. Another example aspect of the present disclosure is directed to a computing system. The system can include one or more processors and one or more non-tran