Search

US-20260129048-A1 - SYSTEM AND METHOD FOR INTEGRATION OF DATA DRIVEN AGENTS WITH LARGE LANGUAGE MODELS

US20260129048A1US 20260129048 A1US20260129048 A1US 20260129048A1US-20260129048-A1

Abstract

A system and method for effective integration of data driven agents with Large Language Models (LLMs) is provided. The present invention provides for seamless integration of agents with LLMs by allowing easy specification and fine-tuning of granularity of agent roles as well as the types of LLMs suitable for each agent's task. The present invention enables specifying the LLMs, therefore allowing to choose LLMs for adequate task performance. The present invention effectively integrates LLM agents and tools from multiple providers. The present invention discloses a system and a method for safe data usage without any data privacy issue.

Inventors

  • Daniel Fink
  • Babak Hodjat

Assignees

  • Cognizant Technology Solutions U.S. Corporation

Dates

Publication Date
20260507
Application Date
20250310

Claims (20)

  1. 1 . A system for facilitating single point of user access to multiple large language models to generate a response to a user query from a access, the system comprising: a first network of agents, wherein at least two agents facilitate access to at least two different large language models; and an interface for receiving a user's query and initiating a session with the first network of agents, wherein initiating the session includes creating an instance of a front man agent having access to a first large language model and further wherein the front man agent has access to at least a first branch agent, the first branch agent having access to a second large language model.
  2. 2 . The system according to claim 1 , wherein the front man agent and the first branch agent have access to one or more coded tool nodes.
  3. 3 . The system according to claim 2 , wherein the front man agent processes the user's query against the first large language model and determines that access to the first branch agent and the second large language model is necessary to answer the user's query.
  4. 4 . The system according to claim 1 , wherein the system includes a dictionary of user data which is designated as private by the user and further wherein, when the user's query includes at least some data which is designated as private, the system applies pre-established rules regarding use of the private data in one or more downstream actions.
  5. 5 . The system according to claim 1 , further comprising a registry of individual agents available to the user via the single point of user access, wherein the registry includes at least a list of each individual agent in the first network of agents.
  6. 6 . The system according to claim 5 , wherein the registry includes a listing of data-only dictionaries of each individual agent specification for each individual agent in the first network, the dictionaries being in the form of a key/value mapping.
  7. 7 . The system according to claim 6 , wherein the data-only dictionaries of each agent specification may be read from one or more files selected from the following text-based formats: JSON (JavaScript Object Notation), Hocon (Human-Optimized Configuration Object Notation), YAML (yet another markup language), or XML (Extensible Markup Language).
  8. 8 . The system according to claim 1 , wherein communications between the front man agent and at least a first branch agent is via chat streaming and further wherein chat details are stored in a journal at both the front man agent and the first branch agent.
  9. 9 . The system according to claim 6 , wherein the registry of agents further includes one or more dictionaries of the agent specification for one or more external agents accessible to the user via the single point of user access, wherein the one or more external agents are located in a different network of agents from the first network of agents.
  10. 10 . A process for generating a response to a user query from a single point of user access to a data-driven network of agents, the process comprising: receiving, at the single point of user access to the data-driven network, the user query; initiating an agent session between the user and the data-driven network based on user input in the user query; creating an instance of a front man agent having access to a first large language model; processing, by the first large language model, the user input and determining that additional processing is required to answer the user query; accessing, by the front man agent, instantiation instructions in an agent tool registry at the front man agent, for a first branch agent having access to a second large language model; creating an instance of the first branch agent having access to the second large language model; performing, by the second large language model, the additional processing in accordance with first chat input from the front man agent; returning, by the first branch agent, a first output chat response to the front man agent based on the additional processing; and determining, by the front man agent, that a response to the user query is complete and communicating the completed response to the user via the single point of user access to the data-driven network of agents.
  11. 11 . The process according to claim 10 , further comprising wherein the front man agent further determines that second additional processing is required to answer the user query; accessing, by the front man agent, invocation instructions in the agent tool registry at the front man agent, for a first coded tool node for performing the second additional processing; invoking the first coded tool node; performing, by the first coded tool node, the second additional processing in accordance with second chat input from the front man agent; returning, by the first coded tool node, a second output chat response to the front man agent based on the second additional processing; and determining, by the front man agent, that a response to the user query is complete and communicating the completed response to the user via the single point of user access to the data-driven network of agents.
  12. 12 . The process according to claim 10 , further comprising: determining, by the system, that at least part of the user data in the user query is designated as private data; and applying one or more pre-established rules regarding use of the private data by the first or second large language models.
  13. 13 . The process according to claim 10 , wherein the agent tool registry is provided in the instantiation of the first branch agent having access to the second large language model.
  14. 14 . The process according to claim 13 , wherein the agent tool registry includes a listing of data-only dictionaries of each agent specification for each agent in the data-driven network, the dictionaries being in the form of a key/value mapping.
  15. 15 . The process according to claim 14 , wherein the data-only dictionaries of each agent specification is read from one or more files selected from the following text-based formats: JSON (JavaScript Object Notation), Hocon (Human-Optimized Configuration Object Notation), YAML (yet another markup language), or XML (Extensible Markup Language).
  16. 16 . The process according to claim 1 , wherein communications between the front man agent and the first branch agent is via chat streaming and further wherein chat details from the communications are stored in a journal at both the front man agent and the first branch agent.
  17. 17 . The process according to claim 10 , wherein the agent tool registry of agents further includes one or more dictionaries of the agent specification for one or more external agents accessible to one or both of the front man agent and the first branch agent, wherein the one or more external agents are located in an external data-driven network of agents.
  18. 18 . A system for facilitating single point of user access to a data-driven agent network to generate a response to a user query, the system comprising: a user interface for receiving a user's query and initiating a session with the data-driven agent network, wherein the initiated session includes, an agent tool registry listing each individual data-driven agent in the data-driven agent network and an individual agent specification for each individual data-driven agent, and a designation of a front man agent including a first large language model, for performing initial processing of the user query to determine a response thereto; and multiple tools, wherein at least one of the multiple tools is a branch agent including a second large language model for performing additional processing of the user query to determine a response thereto, and further wherein the front man agent is in either direct or indirect in communication with each of the multiple tools via a data-driven chat streaming session.
  19. 19 . The system of claim 18 , wherein each individual data-driven agent specification is a data-only dictionary being in the form of a key/value mapping.
  20. 20 . The system according to claim 19 , wherein the data-only dictionaries of each data-driven agent specification is read from one or more files selected from the following text-based formats: JSON (JavaScript Object Notation), Hocon (Human-Optimized Configuration Object Notation), YAML (yet another markup language), or XML (Extensible Markup Language).

Description

CROSS-REFERENCE TO RELATED APPLICATIONS The present application claims benefit of priority to U.S. Provisional Patent Application No. 63/714,906 entitled SYSTEM AND METHOD FOR INTEGRATION OF DATA DRIVEN AGENTS WITH LARGE LANGUAGE MODELS filed Nov. 1, 2024, which is incorporated herein by reference in its entirety. BACKGROUND OF THE EMBODIMENTS Field of the Invention The present invention relates generally to the field of Large Language Models (LLMs). More particularly, the present invention relates to a system and a method for instantiating and networking multi-agent systems, including multiple data driven LLM agents, in order to perform complex tasks. Description of Related Art Different types of Generative Artificial Intelligence (Gen AI) based Large Language Models (e.g., Open AIR, Anthropic®, etc.), are being implemented for ingesting inputs as structured functions (e.g., user prompts, including natural language prompts) to provide a desired output, e.g., new content, such as answers to questions. LLMs can be described as a reasoning engine or cognitive component of a Gen AI system. To facilitate access to the LLM engine by a user, constructs, referred to as agents (also called assistants) have been developed to integrate with an LLM. These agents are code-implemented layers on top of the LLM that observe and interact with LLM based on previous conversations, allowing it to plan and respond iteratively to a user to achieve natural-language goals. The existing art also supports facilitating access by the agent to coded-tools, such as math calculations, web API calls to access a web service, and other functions that LLMs typically do not do well, in order to manage and generate the responses to the user. FIG. 1 exemplifies an existing agent-based single LLM-access system which is supported by the LangChain open-source applications development framework and toolset (library). The agent in FIG. 1 has its own prompts and instructions, but it runs into various limitations. For example, it is limited to access to a single LLM and certain defined tools, and thus will be limited in what it can do, i.e., knowledge-wise and/or time-wise. Different LLMs have varying strengths and weaknesses, costs, and limitations on input/output capacity, affecting their task performance. It would stand to reason that access to multiple LLMs would improve complex task completion. Accessing multiple LLMs—or having the ability to access multiple LLMs—to complete a complex task is not so easy to implement. Ideally, a user would simply be able to submit their query/task in natural language through a single interface and have automatic access to the knowledge of multiple LLMs. One way to implement such access is to use multiple agents, each agent having a different LLM as its core engine. But, in order to implement multi-agent, multi-LLM scenarios, there are many considerations, including, but not limited to, determining level and granularity of agent roles, types of LLMs (e.g., proprietary), agent integrations with tools, data privacy, limits on interoperability between agents supported by different providers, etc. Further, implementation requires coding experience. Current offerings involve creating single- or multi-agent systems through specifying the aspects of the agents and stitching these together via code in a language like Python. Deployment of such systems are often subject to lengthy security scans of the software to ensure practices are up-to-date, i.e., as un-hackable as possible, before they can be used by internal corporate entities or outside clients. These are all the tasks of a programmer, yet agent network creation can be at its most creative in the hands of subject-matter experts, who are not necessarily programmers. Exemplary descriptions of multi-agent systems include: Wu, et. al., AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, arXiv: 2308.08155v2 [cs.AI] 3 Oct. 2023; LangGraph's agent architectures including multi-agent systems and Streamlit's single API to multiple LLM product offering. None of these descriptions enable a single-point of access to multiple LLMs to solve a complex problem in a networked fashion. That is, AutoGen and LanGraph refer to a singular LLM agent in the examples provided. While they do describe other non-LLM agents, e.g., search engine, web scraper, as being part of their multi-agent systems, there is no description of a network of agents that provides access to multiple LLM agents. And while Streamlit facilitates access to multiple LLMs, each LLM must be pre-selected by a user before accepting a query/input. There is a need for a system and a method which provides a multi-agent framework for single-point, provider-agnostic, data-secure user access to multiple LLMs. There is a need for a no-code/low-code system that allows for deployment of multi-agent systems without sacrificing the base-level security of code deployment. And there is a need for a system and me