KR-20260066049-A - Evolving Prompts for Neural Networks

Abstract

A system, method, and computer program code for generating task prompts to be included in model inputs to control the model to perform specific tasks. An evolutionary process is used to evolve prompts that progressively improve the extraction of knowledge from a machine learning model. Some machine learning models, such as large-scale language models, can consume significant computing resources, and the implementation of the described technique is configured to use parallel processing to efficiently utilize these resources.

Inventors

• 페르난도 크리산타 토머스
• 바나르세 딜런 수닐
• 미하레프스키 헨리크
• 오신데로 사이먼
• 록타셸 팀

Assignees

• 지디엠 홀딩 엘엘씨

Dates

Publication Date

20260512

Application Date

20240930

Priority Date

20230928

Claims (20)

1. A computer-implemented method for generating a task-prompt to be included in a neural network input for controlling a neural network to perform a task, wherein the method The method includes the step of performing an evolutionary process using first and second processes, wherein the evolutionary process includes a plurality of evolutionary rounds performed on a population, and The above first process is Providing a task-prompt mutation input as an input to a task-prompt mutation neural network to generate a new task-prompt—the task-prompt mutation input comprises a mutation-prompt containing a command for generating a new task-prompt based on at least one individual of the population—; The above second process is Providing a variant-prompt variant input as an input to a variant-prompt variant neural network to generate a new variant-prompt - said variant-prompt variant input includes a variant-prompt and a hypervariable-prompt including a command for modifying said variant-prompt - including; The step of performing the above evolution process is, For at least some of the above evolution rounds, A step of performing the first process to generate a new task prompt to replace the task prompt in the above population; For at least some of the above evolution rounds, A step of performing the second process to generate a new variant prompt to be used in performing the first process; A step of evaluating the suitability of task-prompts for at least some individuals of the above population; and A method comprising the step of selecting a task prompt from the population based on the individual suitability of the task prompt.
2. In claim 1, for at least some of the evolution rounds, the step of performing the first process A step of selecting an individual from the above population; and A method comprising the step of including the task-prompt of the selected object in the task-prompt variant input to generate an evolved task-prompt.
3. In paragraph 2, the step of selecting an individual from the above population is, A method comprising the steps of comparing two or more individuals from the above population and selecting the individual including a task prompt having the highest fitness of the selected individual.
4. In paragraph 3, A method further comprising the step of replacing at least one individual with lower fitness among the two or more individuals containing task-prompts in the above population with a new individual containing the evolved task-prompt.
5. A method according to any one of claims 1 to 4, wherein at least some individuals within the population further comprise a variant-prompt.
6. In paragraph 5, the step of performing the second process for at least some of the evolution rounds is A step of selecting an individual from the above population; and A method comprising the step of including the variant-prompt of the selected individual in the variant-prompt variant input to generate an evolved variant-prompt.
7. In paragraph 6, the step of selecting an individual from the above population is, A method comprising the steps of comparing two or more individuals from the above population and selecting the individual including a task prompt having the highest fitness of the selected individual.
8. In Paragraph 7, A method further comprising the step of replacing the individual having the lowest fitness task-prompt in the above population with a new individual having the evolved variant-prompt.
9. A method in which, in claim 8 subject to claim 4, the new entity comprises the evolved task-prompt and the evolved mutation-prompt.
10. In any one of claims 1 to 9, the method further includes the step of initializing a population of the individual before performing the evolution process, and the step of initializing the population is A step of selecting an initial variation prompt from a set of variation prompts; A step of including the initial variation-prompt in the prompt initialization input to a prompt initialization neural network to generate an initial task-prompt; and A method comprising the step of adding an object containing the initial task-prompt to the above population.
11. In item 10, the step of initializing the above population is, A step of selecting a modifier from a set of modifiers; and A method comprising the step of including the modifier in the prompt initialization input to the prompt initialization neural network to generate the above initial task-prompt.
12. In claim 10 or 11, the step of initializing the population is, A method comprising the step of including the task specification in the prompt initialization input to the prompt initialization neural network to generate the above initial task prompt.
13. In any one of paragraphs 1 through 12, For at least some of the above evolution rounds, a step of maintaining a set of one or more individuals having the highest fitness in the corresponding evolution round; and For at least some of the above evolution rounds, A method further comprising the step of performing the first process, which includes including the task-prompt of the set of one or more objects in the task-prompt variation input.
14. In paragraph 13, the task-prompt of the set of one or more objects is included in the task-prompt variation input in chronological order.
15. In any one of claims 1 to 14, with respect to at least some of the evolution rounds, The step of performing the above-mentioned first process is, The step of including a list of task prompts in the above task prompt variant input; and A method comprising the step of including a variation-prompt configured to cause the task-prompt variation neural network to continue the list with one or more task-prompts in the task-prompt variation input.
16. In paragraph 15, the step of generating a list of the above task prompts is further included, and the step of generating the list is A method comprising the step of filtering the objects of the population to remove objects having task prompts within a threshold similarity of task prompts of other objects of the population.
17. A method according to claim 15 or 16, wherein the list of task-prompts is included in the task-prompt variation input in a random order.
18. A method according to claim 15 or 16, wherein the list of task-prompts is included in the task-prompt variation input in order of suitability.
19. A method according to claim 18, further comprising the step of including an order indication indicating the order of the list of task-prompts in the task-prompt variant input.
20. In any one of claims 1 to 19, when performing the first process for at least some of the evolution rounds, The step of providing the task-prompt variant input comprising at least one entity includes the step of including at least one inference path of the entity that resulted in the correct answer for at least one training example in the task-prompt variant input; and A method comprising the step of providing the task-prompt variant input including the variant-prompt, wherein the step of including the variant-prompt configured to instruct the task-prompt variant neural network to generate a task-prompt leading to the inference path in the task-prompt variant input.

Description

Evolving Prompts for Neural Networks Cross-reference regarding related applications This application claims priority to U.S. Provisional Application No. 63/586,323, filed on September 28, 2023. The disclosure of the said prior application is construed as part of this application and is incorporated by reference into the disclosure of this application. This specification relates to processing data using machine learning models. Machine learning models receive input and generate an output, for example, a predicted output, based on the received input. Some machine learning models are parametric models that generate an output based on the received input and the model's parameter values. Some machine learning models are deep models that employ multiple layers to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that generate an output by applying a non-linear transformation to each received input. Figure 1 illustrates an example of a prompt evolution system. Figure 2 is a flowchart of an exemplary method for generating a task prompt. Figure 3 illustrates an initialization process for filling a population. Figure 4 is a flowchart of another exemplary method for generating a task prompt. Figure 5 is a flowchart of a method for performing a task. Figure 6 illustrates the population fit of task-prompts for a large number of evolution rounds. Similar reference numbers and instructions in various drawings indicate similar elements. Prompting for a model, e.g., a large-scale language model (LLM), is critical to its downstream performance. A prompt evolution system and associated methods that provide a mechanism for evolving and constructing prompts for a given domain are described herein. A population of task prompts (e.g., prompts for tasks in a specific domain) may be initialized, mutated, and evaluated for fitness over multiple generations to evolve the task prompts. According to the method and system described herein, variation of the task prompt is governed by variation prompts that the system also generates, mutates, and evolves in a self-referential manner. In this way, the mechanism allows for the improvement of the task prompt and also the improvement of the variation prompts that improve the task prompt. A model trained to perform a task in a specific domain may be trained using training data that includes a batch of question (Q) and answer (A) pairs. According to the examples described herein, during inference, the model may be provided with a task prompt P, which is a string used to precondition the context ahead of some additional input question Q. The task prompt P is intended to ensure that a better (e.g., more accurate) answer A is provided than when question Q is presented without the prompt P. The prompt evolution system and associated method described herein enable the generation and variation of task prompt P according to an evolutionary algorithm by conditioning the evolution to a variation prompt M. Additionally, the prompt evolution system and associated method described herein also enable the generation and variation of variation prompt M. By evolving the variation prompt M, the system is self-referential in that it not only generates an improved task prompt but also improves the method of generating the improved task prompt itself. FIG. 1 illustrates an example of a prompt evolution system (100). The prompt evolution system (100) is an example of a system implemented as a computer program on one or more computers in one or more locations where the systems, components, and techniques described below can be implemented. The prompt evolution system (100) includes a task-prompt variation model (110). The task-prompt variation model (110) is configured to receive a task-prompt variation input (112) and generate a new task-prompt (114). The process performed by the task-prompt variation model (110) may be referred to as a first process (111). The task-prompt variation input (112) may include a task-prompt P and/or a variation-prompt M, but the origin of these task-prompt P and variation-prompt M may vary according to an exemplary embodiment as described in more detail below. In some embodiments, the task-prompt variation model (110) is an LLM. Considering that variations of input text can be generated using an LLM, the task-prompt variation input (112) includes an (initial) task-prompt P and a variation prompt M. The task-prompt variation model (110) performs an evolution algorithm by generating a variationd task-prompt P' conditioned to the variation prompt M based on the initial task-prompt P. The task-prompt variation model (110) outputs the variationd task-prompt P' as a new task-prompt (114). The prompt evolution system (100) also includes a mutation-prompt mutation model (120). The mutation-prompt mutation model (120) is configured to receive a mutation-prompt mutation input (1