Search

US-20260127391-A1 - LANGUAGE MODEL BASED RULE DETECTION, IMPROVEMENT, AND APPLICATION MECHANISM

US20260127391A1US 20260127391 A1US20260127391 A1US 20260127391A1US-20260127391-A1

Abstract

The invention relates to a system for developing and designing an AI service, which comprises a task exploration module, a demand analysis module, an AI service construction module and an AI service deployment module; the task exploration module acquires a task background and sends the task background to the demand analysis module after recording; the demand analysis module asks a question to the user according to the initial task description, and updates the initial task description according to the user answer to generate an AI chain frame; the AI service construction module generates a corresponding AI chain based on the AI chain frame and the task background; and the AI service deployment module assembles the working units corresponding to each task step in a serial or parallel mode to form an AI chain, and finally realizes the operation of the AI chain based on each working unit. According to the method and the system, the problem can be solved according to the initial task description, task requirements are more effectively understood, and the AI chain meeting the requirements of clients is generated.

Inventors

  • Maxime Defauw
  • Sriram Krishnan

Assignees

  • Palantir Technologies Inc.

Dates

Publication Date
20260507
Application Date
20250515

Claims (20)

  1. 1 . A computer-implemented method, performed by a computing system having one or more hardware computer processors and one or more computer-readable storage devices storing software instructions executable by the computing system, the computer-implemented method comprising: receiving or accessing one or more data transformations, wherein the one or more data transformations include initial data states and respective corresponding transformed data states; generating a first artificial intelligence (“AI”) model prompt, the first AI model prompt including the one or more data transformations and rule generation instructions, wherein the rule generation instructions include instructions to generate one or more transformation rules based on comparisons between the initial data states and respective corresponding transformed data states of the one or more data transformations; providing the first AI model prompt to a first AI model; receiving a first output from the first AI model in response to the first AI model prompt; determining and/or parsing the first output from the first AI model to determine the one or more transformation rules; receiving or accessing a first initial data state to be transformed; generating a second AI model prompt, the second AI model prompt including the one or more transformation rules and data transformation instructions, wherein the data transformation instructions include instructions to generate a first transformed data state by applying at least one of the one or more transformation rules to the first initial data state; providing the second AI model prompt to a second AI model; receiving a second output from the second AI model in response to the second AI model prompt; and determining the first transformed data state from the second output.
  2. 2 . The computer-implemented method of claim 1 , wherein the first AI model and/or the second AI model comprise at least one of: language models, or large language models (“LLM”).
  3. 3 . The computer-implemented method of claim 1 , wherein the first AI model and the second AI model are at least one of: different AI models, or a same AI model.
  4. 4 . The computer-implemented method of claim 1 , wherein the rule generation instructions further include instructions to avoid generating duplicate or similar transformation rules.
  5. 5 . The computer-implemented method of claim 1 , wherein the rule generation instructions further include instructions to indicate, for each transformation rule of the one or more transformation rules, whether the transformation rule is a contradiction to another of the one or more transformation rules.
  6. 6 . The computer-implemented method of claim 1 , wherein the rule generation instructions further include instructions to output the one or more transformation rules in a computer parseable format.
  7. 7 . The computer-implemented method of claim 1 , further comprising: receiving one or more user inputs approving or rejecting transformation rules of the one or more transformation rules, wherein any rejected transformation rules are removed from the one or more transformation rules such that the second AI model prompt does not include such rejected transformation rules.
  8. 8 . The computer-implemented method of claim 1 , further comprising: generating data usable to generate, and/or causing display of, an interactive graphical user interface including: a listing of the one or more transformation rules, and interactive user interface elements usable for a user to approve or reject transformation rules.
  9. 9 . The computer-implemented method of claim 8 , wherein the interactive graphical user interface further includes: for a selected transformation rule of the one or more transformation rules, indications of any other transformation rules of the transformation rules that are similar to the selected transformation rule.
  10. 10 . The computer-implemented method of claim 1 , wherein the data transformation instructions further include feedback from a previous translation of the first initial data state and instructions to take into account the feedback in generating the first transformed data state.
  11. 11 . The computer-implemented method of claim 1 , wherein the second AI model prompt further includes one or more example data transformations that are similar to the first initial data state.
  12. 12 . The computer-implemented method of claim 1 , wherein the rule generation instructions further include instructions to indicate, for each of the transformation rules of the one or more transformation rules, respective associated confidence scores.
  13. 13 . The computer-implemented method of claim 12 , wherein the data transformation instructions further include instructions to prioritize the one or more transformation rules based on the respective associated confidence scores.
  14. 14 . The computer-implemented method of claim 12 , further comprising: receiving one or more user inputs approving, rejecting, modifying, and/or providing feedback regarding, the first transformed data state; and in response to and/or based on the one or more user inputs, modifying a confidence score associated with a transformation rule, of the one or more transformation rules, that was used to generate the first transformed data state.
  15. 15 . The computer-implemented method of claim 14 , wherein the one or more user inputs are used as feedback in a subsequent translation of the first initial data state.
  16. 16 . The computer-implemented method of claim 12 further comprising: determining a correctness of the first transformed data state; and based on the correctness determination at least one of: modifying a confidence score associated with a transformation rule, of the one or more transformation rules, that was used to generate the first transformed data state; or generating a feedback to be used in a subsequent translation of the first initial data state.
  17. 17 . The computer-implemented method of claim 16 , wherein determining the correctness of the first transformed data state includes at least one of: parsing the first transformed data state, validating the first transformed data state, or executing the first transformed data state.
  18. 18 . The computer-implemented method of claim 1 , further comprising: generating data usable to generate, and/or causing display of, an interactive graphical user interface including: an indication of the first initial data state; an indication of the first transformed data state; and interactive user interface elements usable for a user to approve, reject, modify, and/or provide feedback regarding, the first transformed data state.
  19. 19 . A system comprising: one or more computer-readable storage mediums or devices comprising, configured to store, and/or storing program instructions; and one or more processors configured to execute the program instructions to cause the system to perform the computer-implemented method of claim 1 .
  20. 20 . One or more computer-readable storage mediums or devices comprising, configured to store, and/or storing program instructions, the program instructions executable by one or more processors to cause the one or more processors to perform the computer-implemented method of claim 1 .

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims benefit of U.S. Provisional Patent Application No. 63/715,062, filed Nov. 1, 2024, and titled “LANGUAGE MODEL BASED RULE DETECTION, IMPROVEMENT, AND APPLICATION MECHANISM” and U.S. Provisional Patent Application No. 63/729,882, filed Dec. 9, 2024, and titled “LANGUAGE MODEL-BASED RULE DETECTION, IMPROVEMENT, AND APPLICATION.” The entire disclosures of each of the above items are hereby made part of this specification as if set forth fully herein and incorporated by reference for all purposes, for all that they contain. Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet as filed with the present application are hereby incorporated by reference under 37 CFR 1.57 for all purposes and for all that they contain. TECHNICAL FIELD The present disclosure relates to systems and techniques for utilizing computer-based models. More specifically, various implementations of the present disclosure relate to computerized systems and techniques for using, e.g., large language models to detect, improve, and apply rules. BACKGROUND The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Computers can be programmed to perform calculations and operations utilizing one or more computer-based models. For example, language models can be utilized to produce text-based outputs based on inputs given to the language models. SUMMARY The systems, methods, and devices described herein each have several aspects, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of this disclosure, several non-limiting features will now be described briefly. The present disclosure describes various aspects of transforming computer-based information, such as data, from one state to another. A data state can refer to data in a particular state, a particular form, and/or at a particular point in time. Transforming data from an initial data state to a transformed data state can present several technical challenges. For example, data transformations can include transforming a large amount of data together. Further, complex data transformations may require specialized knowledge (also referred to as domain-specific knowledge). Computer-based models can be used by users to solve various problems. For example, artificial intelligence (“AI”) models, such as language models, large language models (“LLMs”) and/or other AI models, can be useful for data processing, including receiving natural language prompts and providing responses based on data on which the AI model is trained. While other types of AI models may be used in various implementations, for convenience, reference will be made to the use of LLMs in the present disclosure. LLMs can be used by users to perform various tasks, such as generating, translating, summarizing, and/or otherwise interacting with or processing text based on prompts given to the LLMs. However, using LLMs to perform specialized and repetitive tasks, such as data transformations, may present several technical challenges. For instance, LLMs may generally be trained on broad datasets and may lack the domain-specific knowledge to consistently perform the specialized and repetitive tasks. Further, the output of an LLM may not be determined solely based on an input to the LLM (e.g., the LLM may be nondeterministic). As such, each repetition of a task using an LLM may produce inconsistent results, which may not be desirable for some tasks and may increase the amount of oversight needed to use LLMs for such tasks. The present disclosure describes systems and methods (generally referred to herein as a “system”) that can, according to various implementations, advantageously overcome various of the technical challenges mentioned above, among other challenges. For example, various implementations of the systems and method of the present disclosure can employ a workflow (including various graphical user interfaces (“GUIs”)) for transforming data from an initial data state to a transformed data state using one or more interactions with LLMs. The system may allow for the generation of one or more transformation rules that can provide structure and/or domain-specific knowledge, increasing the reliability of transformations generated by the LLMs. As will be described in more detail below, the transformation rules may be used by an LLM (e.g., given in prompts to the LLM) along with data in initial states, to transform the data to transformed states in a consistent manner. According to various implementations, the data transformed by the system can include any textual data, such as computer