EP-4740141-A1 - EVOLVING PROMPTS FOR NEURAL NETWORKS

Abstract

Systems, methods, and computer program code for generating a task-prompt for inclusion in a model input for controlling a model to perform a task. An evolutionary process is used to evolve prompts that increasingly improve the extraction of knowledge from a machine learned model. Some machine learned model, such as Large Language Models, can consume significant computing resources, and implementations of the described techniques are configured to use parallel processing in a way that facilitates makes efficient use of these resources.

Inventors

• FERNANDO, Chrisantha Thomas
• BANARSE, DYLAN SUNIL
• MICHALEWSKI, Henryk
• OSINDERO, Simon
• ROCKTÄSCHEL, Tim

Assignees

• GDM Holding LLC

Dates

Publication Date

20260513

Application Date

20240930

Claims (20)

1. 1. A computer-implemented method for generating a task-prompt for inclusion in a neural network input for controlling a neural network to perform a task, the method comprising: performing an evolutionary process using first and second processes, the evolutionary process comprising a plurality of evolutionary rounds performed on a population, the first process, comprising: providing a task-prompt mutation input as input to a task-prompt mutation neural network to generate a new task-prompt, the task-prompt mutation input comprising a mutation-prompt comprising an instruction to generate a new taskprompt based on at least one individual of the population; the second process, comprising: providing a mutation-prompt mutation input as input to a mutation-prompt mutation neural network to generate a new mutation-prompt, the mutation-prompt mutation input comprising a mutation-prompt and a hypermutation-prompt comprising an instruction to modify the mutation-prompt; wherein performing the evolutionary process comprises: for at least some of the evolutionary rounds: performing the first process to generate a new task-prompt to replace a taskprompt in the population; for at least some of the evolutionary rounds: performing the second process to generate a new mutation-prompt for use in performing the first process; evaluating a fitness of task-prompts of at least some individuals of the population; and selecting a task-prompt from the population based on respective finesses of the taskprompts.
2. 2. The method claim 1, wherein for at least some of the evolutionary rounds, performing the first process comprises: selecting an individual from the population; and including in the task-prompt mutation input the task-prompt of the selected individual to generate an evolved task-prompt.
3. 3. The method of claim 2, wherein selecting an individual from the population comprises: comparing two or more individuals from the population and selecting the individual comprising a task-prompt with the highest fitness of the selected individuals.
4. 4. The method of claim 3, further comprising: replacing in the population at least one of the two or more individuals comprising a task-prompt with a lower fitness with a new individual comprising the evolved task-prompt.
5. 5. The method of any preceding claim, wherein at least some individuals in the population further comprise a mutation-prompt.
6. 6. The method of claim 5, wherein for at least some of the evolutionary rounds, performing the second process comprises: selecting an individual from the population; and including in the mutation-prompt mutation input the mutation-prompt of the selected individual to generate an evolved mutation-prompt.
7. 7. The method of claim 6, wherein selecting an individual from the population comprises: comparing two or more individuals from the population and selecting the individual comprising a task-prompt with the highest fitness of the selected individuals.
8. 8. The method of claim 7, further comprising: replacing in the population the individual comprising a task-prompt with a lower fitness with a new individual comprising the evolved mutation-prompt.
9. 9. The method of claim 8 as dependent on claim 4, wherein the new individual comprises the evolved task-prompt and the evolved mutation-prompt.
10. 10. The method of any preceding claim, further comprising initializing the population of individuals prior to performing the evolutionary process, comprising: selecting an initial mutation-prompt from a set of mutation-prompts; including the initial mutation-prompt in a prompt initialization input to a prompt initialization neural network to generate an initial task-prompt; and adding an individual comprising the initial task-prompt to the population.
11. 11. The method of claim 10, wherein initializing the population comprises: selecting a modifier from a set of modifiers; and including the modifier in the prompt initialization input to the prompt initialization neural network to generate the initial task-prompt.
12. 12. The method of claim 10 or 11, wherein initializing the population comprises: including a task specification in the prompt initialization input to the prompt initialization neural network to generate the initial task-prompt.
13. 13. The method of any preceding claim, further comprising: maintaining, for at least some of the rounds of evolution, a set of one or more individuals having a highest fitness in that round of evolution; and for at least some of the evolutionary rounds: performing the first process comprises including the task-prompts of the set of one or more individuals in the task-prompt mutation input.
14. 14. The method of claim 13, wherein the task-prompts of the set of one or more individuals is included in the task-prompt mutation input in chronological order.
15. 15. The method of any preceding claim, wherein for at least some of the evolutionary rounds: performing the first process comprises: including a list of task-prompts in the task-prompt mutation input; and including in the task-prompt mutation input a mutation-prompt configured to cause the task-prompt mutation neural network to continue the list with one or more task-prompts.
16. 16. The method of claim 15, further comprising generating the list of task-prompts, comprising: filtering the individuals of the population to remove individuals having a task-prompt that is within a threshold similarity of a task-prompt of another individual of the population.
17. 17. The method of claim 15 or 16, wherein the list of task-prompts is included in the taskprompt mutation input in a random order.
18. 18. The method of claim 15 or 16, wherein the list of task-prompts is included in the taskprompt mutation input in order of fitness.
19. 19. The method of claim 18, further comprising including an order indication in the taskprompt mutation input indicating an order of the list of task-prompts.
20. 20. The method of any preceding claim, wherein for at least some of the evolutionary rounds when performing the first process: providing the task-prompt mutation input comprising at least one individual comprises including in the task-prompt mutation input at least one reasoning path of the individual that resulted in a correct answer on at least one training example; and providing the task-prompt mutation input comprising a mutation-prompt comprises including in the task-prompt mutation input a mutation-prompt configured to instruct the task-prompt mutation neural network to generate a task-prompt that led to the reasoning path.

Description

EVOLVING PROMPTS FOR NEURAL NETWORKS CROSS-REFERENCE TO RELATED APPLICATION [0001] This application claims priority to U.S. Provisional Application No. 63/586,323, filed on September 28, 2023. The disclosure of the prior application is considered part of and is incorporated by reference in the disclosure of this application. BACKGROUND [0002] This specification relates to processing data using machine learning models. [0003] Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model. [0004] Some machine learning models are deep models that employ multiple layers of models to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output. SUMMARY [0005] This specification generally describes a system implemented as computer programs on one or more computers in one or more locations that enable a user to evolve task-prompts for input to a (trained) machine learning model, such as a Large Language Model (LLM). Some models, such as LLMs, already encode knowledge about a vast array of problems. However, it is not always straightforward to use or extract that knowledge. The techniques described herein provide a mechanism to evolve prompts that increasingly improve the extraction of knowledge from a machine learned model. Further, processing using Large Language Models can consume significant computing resources and some implementations of the described techniques are configured to facilitate parallel processing in a way that makes efficient use of these resources. [0006] In one example, a computer-implemented method for generating a task-prompt for inclusion in a model input for controlling a model to perform a task comprises performing an evolutionary process using first and second processes. The evolutionary process comprises a plurality of evolutionary rounds performed on a population. The model may be a neural network. The model may be a trained model (e.g. a trained neural network) trained to perform the task. [0007] The first process comprises providing a task-prompt mutation input as an input to a task-prompt mutation model to generate a new task-prompt. The task-prompt mutation input comprises a mutation-prompt comprising an instruction to generate a new task-prompt based on at least one individual of the population. Generating a new task-prompt based on an individual of the population may mean generating a new task-prompt based on any property of, or associated with, the at least one individual. The at least one individual of the population may be an individual of the current population or of a previous population. The task-prompt mutation model may be a neural network, referred to as a task-prompt mutation neural network. The task-prompt mutation model may be a trained model, e.g. a trained task-prompt mutation neural network. The new task-prompt may be referred to as an evolved task-prompt. [0008] The second process comprises providing a mutation-prompt mutation input as input to a mutation-prompt mutation model to generate a new mutation-prompt. The mutation-prompt mutation input comprises a mutation-prompt and a hypermutation-prompt comprising an instruction to modify the mutation-prompt. The mutation-prompt may be referred to as a mutation operator herein. The mutation-prompt mutation model may be a neural network, referred to as a mutation-prompt mutation neural network. The mutation-prompt mutation model may be a trained model, e.g. a trained mutation-prompt mutation neural network. The new mutation-prompt may be referred to as an evolved mutation-prompt. [0009] Performing the evolutionary process comprises performing, for at least some of the evolutionary rounds, the first process to generate a new task-prompt to replace a task-prompt in the population. Performing the evolutionary process also comprises performing, for at least some of the evolutionary rounds, the second process to generate a new mutation-prompt for use in performing the first process. Performing the evolutionary process further comprises evaluating a fitness of task-prompts of at least some individuals of the population and selecting a task-prompt from the population based on respective finesses of the task-prompts. [0010] In another example, a computer-implemented method for generating a task-prompt for inclusion in a model input for controlling a model to perform a task comprises receiving a task specification and determining, based on the task specification, an initial task-prompt. The model may be a neural network. The model may be a trained model (e.g. a trained neural network) trained to perform the task. [0011] In this examp