US-12620026-B1 - Agentic framework for intent-driven responses in computer-based mortgage systems

Abstract

A method of performing an intention classification with a first large language model included in a first agent included in a multi-agent system, the first large language model configured to receive the input message as a text input, the intention classification associating an intent of the request with a corresponding action. Sending, from the first large language model to an action router, a response including a key that uniquely identifies an action handler from among a plurality of available action handlers, the action handler designated to coordinate operations of the multi-agent system to complete the corresponding action with a workflow defined by the selected action handler. Completing the workflow, at least in part, with the designated action handler generating function calls directed to the multi-agent system and processing API responses received from the multi-agent system to complete tasks included in the predetermined set of tasks.

Inventors

• Adam Carmel
• Jonathan Foy
• Austin M. Hoyle
• David B. Torrance
• Michael G. Ouellette

Assignees

• PollyEx, Inc.

Dates

Publication Date

20260505

Application Date

20250506

Claims (16)

1. 1 . A method generating responses to messages provided by a plurality of users via a chat widget operating on a user computing device, respectively, the messages provided in a natural language and directed to a multi-agent system remote from the respective user computing device, the multi-agent system including a plurality of agents each of the plurality of agents including a respective large language model, the method comprising: on receipt of a message provided by a user included in the plurality of users, the message including a request to perform an action using the multi-agent system, performing acts of: (a) classifying, by one or more processors, with a first large language model included in a first agent included in the multi-agent system, an intent of the request and associating the intent with a corresponding action, the first large language model configured to receive the message as a text input; (b) identifying, with the first large language model, an action handler from among a plurality of available action handlers based on the intent, and sending, from the first large language model, to an action router operating on the user computing device, a response including a key that uniquely identifies the action handler from among a plurality of available action handlers, the action handler designated to handle the request and coordinate operations of the multi-agent system to complete the corresponding action with a workflow defined by the action handler, the workflow including a predetermined set of tasks that are established in advance independent of an operation of the first large language model and independent of an operation of any of the respective large language models, the predetermined set of tasks for completion by a predefined set of agents included in the plurality of agents in a predefined order, where each of the predetermined set of tasks, each of the predefined set of agents and the predefined order, respectively, are established in advance by the action handler for the intent and corresponding action, independent of an operation of the action handler subsequent to the receipt of the message; (c) completing the workflow, at least in part, with the action handler generating function calls directed to the multi-agent system and processing API responses received from the multi-agent system to complete tasks included in the predetermined set of tasks; (d) on a completion of the workflow, sending information output from an agent included in the multi-agent system to the action handler, the action handler routing the information to a response handler operating on the user computing device, the response handler processing the information to generate a conversational output shared with the user via the chat widget; and (e) repeating acts (a)-(d) for each message received from respective users included in the plurality of users, each message included in a plurality of messages received from the plurality of users; for each message included in the plurality of messages for which the intent that is identified at act (a) matches the intent of a different message included in the plurality of messages: identifying the action handler at act (b) as a selected action handler that is the same as identified for others of the different messages having the matching intent, the selected action handler coordinating operations of the multi-agent system to complete the corresponding action with the same workflow defined by the selected action handler, the workflow including the predetermined set of tasks that match those identified for completion for others of the different messages having the matching intent; and for each message included in the plurality of messages for which the intent that is identified at act (a) differs from the intent identified for others of the different messages: identifying the action handler at act (b) that is a different action handler than the selected action handler, the different action handler coordinating operations of the multi-agent system to complete the corresponding action with the workflow defined by the selected action handler, the workflow including the predetermined set of tasks that differ from those identified for completion for the different messages having a different intent.
2. 2 . The method of claim 1 , wherein the action handler is a first action handler, wherein the workflow is a first workflow, wherein the set of predetermined tasks is first set of predetermined tasks, wherein the multi-agent system includes a plurality of agents dedicated to a performance of a specific set of tasks for different workflows, respectively and including a respective large language model, and wherein the method further comprises if the first large language model is unable to identify, with the processing of the text input, an action handler associated with one of the plurality of agents, selecting a second action handler to coordinate operations of a generalist agent of the multi-agent system to respond to the message including a query concerning at least one of a question concerning capabilities of the multi-agent system and a question generally concerning mortgages.
3. 3 . The method of claim 2 , wherein the different workflows are associated with the plurality of agents dedicated to the performance of a specific set of tasks included in workflows established to address user messages concerning at least one of a search of a mortgage rate stack, product eligibility, product pricing, and a saving of a loan scenario.
4. 4 . The method of claim 1 , further comprising fine tuning the first large language model using sets of training data including input messages with queries concerning mortgage product selection, system messages corresponding to the queries and intent classification messages corresponding to the queries.
5. 5 . The method of claim 1 , further comprising providing a front end employed to access the multi-agent system from the user computing device, the front end including each of the action handler in combination with the chat widget, the action router, and the response handler.
6. 6 . The method of claim 1 , further comprising fine tuning at least one language model included in the multi-agent system using training data including system messages to establish the functionality of the large language model, user messages in a form received by the large language model, and response messages output by the large language model.
7. 7 . A system comprising: one or more processors configured to: provide a user computing device with a frontend of a multi-agent system, the multi-agent system including a plurality of agents, the front end including a chat widget, a plurality of action handlers, an action router and a response handler; receive, by the one or more processors, a message provided by a user with the chat widget, the message provided in a natural language and concerning mortgage products; performing, by the one or more processors, an intention classification with a first large language model included in a first agent included in the plurality of agents, the first large language model configured to receive the message as a text input, the intention classification associating an intent of a request provided by the message with a corresponding action; sending, from the first large language model, to an action router operating on the user computing device, a response including a key that uniquely identifies an action handler from among the plurality of action handlers, the action handler designated to coordinate operations of the multi-agent system to complete the corresponding action with a workflow defined by the action handler, the workflow including a predetermined set of tasks, the predetermined set of tasks for completion by a predefined set of agents included in the plurality of agents in a predefined order, where each of the predetermined set of tasks, each of the predefined set of agents and the predefined order, respectively, are established in advance by the action handler for the intent and corresponding action, independent of an operation of the action handler subsequent to the receipt of the message; completing the workflow, at least in part, with the action handler generating function calls directed to the multi-agent system and processing API responses received from the multi-agent system to complete tasks included in the predetermined set of tasks; and on a completion of the workflow, sending information output from an agent included in the plurality of agents to the action handler, the action handler routing the information to a response handler operating on the user computing device, the response handler processing the information to generate a conversational output shared with the user via the chat widget.
8. 8 . The system of claim 7 , wherein the action handler is a first action handler, wherein the workflow is a first workflow, wherein the set of predetermined tasks is first set of predetermined tasks, wherein each of the plurality of agents is dedicated to a performance of a specific set of tasks for different workflows, respectively and including a respective large language model, and wherein the one or more processors is configured to: if the first large language model is unable to identify, with the processing of the text input, an action handler associated with one of the plurality of agents, selecting a second action handler to coordinate operations of a generalist agent of the multi-agent system to respond to the message including a query concerning at least one of a question concerning capabilities of the multi-agent system and a question generally concerning mortgages.
9. 9 . The system of claim 8 , wherein the different workflows are associated with the plurality of agents dedicated to the performance of a specific set of tasks included in workflows established to address user messages concerning at least one of a search of a mortgage rate stack, product eligibility, product pricing, and a saving of a loan scenario.
10. 10 . The system of claim 7 , wherein the processor is configured to fine tune the first large language model using sets of training data including input messages with queries concerning mortgage product selection, system messages corresponding to the queries and intent classification messages corresponding to the queries.
11. 11 . A non-transitory computer readable storage medium storing instructions that when executed by one or more processors causes the one or more processors to perform operations for generating responses to messages provided by a plurality of users via a chat widget operating on a user computing device, respectively, the messages provided in a natural language and directed to a multi-agent system remote from the respective user computing device, the multi-agent system including a plurality of agents each of the plurality of agents including a respective large language model, the operations comprising: on receipt of a message provided by a user included in the plurality of users, the message including a request to perform an action using the multi-agent system, performing acts of: (a) classifying, by one or more processors, with a first large language model included in a first agent included in the multi-agent system, an intent of the request and associating the intent with a corresponding action, the first large language model configured to receive the input-message as a text input; (b) identifying, with the first large language model, an action handler from among a plurality of available action handlers based on the intent, and sending, from the first large language model, to an action router operating on the user computing device, a response including a key that uniquely identifies the action handler from among a plurality of available action handlers, the action handler designated to handle the request and coordinate operations of the multi-agent system to complete the corresponding action with a workflow defined by the action handler, the workflow including a predetermined set of tasks that are established in advance independent of an operation of the first large language model and independent of an operation of any of the respective large language models, the predetermined set of tasks for completion by a predefined set of agents included in the plurality of agents in a predefined order, where each of the predetermined set of tasks, each of the predefined set of agents and the predefined order, respectively, are established in advance by the action handler for the intent and corresponding action, independent of an operation of the action handler subsequent to the receipt of the message; (c) completing the workflow, at least in part, with the action handler generating function calls directed to the multi-agent system and processing API responses received from the multi-agent system to complete tasks included in the predetermined set of tasks; (d) on a completion of the workflow, sending information output from an agent included in the multi-agent system to the action handler, the action handler routing the information to a response handler operating on the user computing device, the response handler processing the information to generate a conversational output shared with the user via the chat widget; and (e) repeating acts (a)-(d) for each message received from respective users included in the plurality of users, each message included in a plurality of messages received from the plurality of users; for each message included in the plurality of messages for which the intent that is identified at act (a) matches the intent of a different message included in the plurality of messages: identifying the action handler at act (b) as a selected action handler that is the same as identified for others of the different messages having the matching intent, the selected action handler coordinating operations of the multi-agent system to complete the corresponding action with the same workflow defined by the selected action handler, the workflow including the predetermined set of tasks that match those identified for completion for others of the different messages having the matching intent; and for each message included in the plurality of messages for which the intent that is identified at act (a) differs from the intent identified for others of the different messages: identifying the action handler at act (b) that is a different action handler than the selected action handler, the different action handler coordinating operations of the multi-agent system to complete the corresponding action with the workflow defined by the selected action handler, the workflow including the predetermined set of tasks that differ from those identified for completion for the different messages having a different intent.
12. 12 . The non-transitory computer readable medium of claim 11 , wherein the action handler is a first action handler, wherein the workflow is a first workflow, wherein the set of predetermined tasks is first set of predetermined tasks, wherein the multi-agent system includes a plurality of agents dedicated to a performance of a specific set of tasks for different workflows, respectively and including a respective large language model, and wherein the operations further comprise if the first large language model is unable to identify, with the processing of the text input, an action handler associated with one of the plurality of agents, selecting a second action handler to coordinate operations of a generalist agent of the multi-agent system to respond to the message including a query concerning at least one of a question concerning capabilities of the multi-agent system and a question generally concerning mortgages.
13. 13 . The non-transitory computer readable medium of claim 12 , wherein the different workflows are associated with the plurality of agents dedicated to the performance of a specific set of tasks included in workflows established to address user messages concerning at least one of a search of a mortgage rate stack, product eligibility, product pricing, and a saving of a loan scenario.
14. 14 . The non-transitory computer readable medium of claim 11 , wherein the operations further comprise fine tuning the first large language model using sets of training data including input messages with queries concerning mortgage product selection, system messages corresponding to the queries and intent classification messages corresponding to the queries.
15. 15 . The non-transitory computer readable medium of claim 11 , further comprising providing a front end employed to access the multi-agent system from the user computing device, the front end including each of the action handler in combination with the chat widget, the action router, and the response handler.
16. 16 . The non-transitory computer readable medium of claim 11 , further comprising fine tuning at least one language model included in the multi-agent system using training data including system messages to establish the functionality of the large language model, user messages in a form received by the large language model, and response messages output by the large language model.

Description

BACKGROUND OF INVENTION 1. Field of Invention This invention relates generally to computer based systems. More specifically, at least one embodiment, relates to an agentic framework for intent-driven responses in computer-based mortgage systems. 2. Discussion of Related Art Today, mortgage industry operations rely heavily on computer-based systems, in part, because of the huge amount of information that must be sorted and processed in a search. The problem has increased over time due to the variations found among mortgage offerings, the number of competing products and the ability for these offerings to slightly differ based on the borrower's qualifications and the property. These computer-based systems allow loan officers to evaluate possible mortgage product options for any given request where it is impractical to perform the analysis in the human mind. However, current approaches suffer from problems that limit their effectiveness. One common approach is to provide these systems with rigid rules-based decision trees that are used in searches for a desired product. This approach lacks the flexibility to properly process queries given the size of the information set and dynamic nature and nuance found in queries. Other current tools are expected to operate in a highly autonomous manner where they are solely responsible for a wide range of complex operations. In practice, these tools provide results that can be unpredictable. For example, large language models (LLMs) are being introduced into mortgage systems. However, LLMs are generally stochastic in nature. That is, there is an element of randomness to their responses even when they receive the same natural language input. In addition, mortgage product and rate data provides a vast, multi-dimensional data set that traditional systems cannot effectively search when presented with a natural language request. These natural language queries are often nuanced because they provide a set of objectives presented in a form that includes objectives that may compete with one another. For example, a user may offer “I need a $5K rebate combined with the lowest rate and a price close to par.” These queries can include a mix of quantitative and qualitative criteria that require logical leaps. Traditional systems struggle with these types of mortgage product and rate data searches queries due to the vast amount of information present in today's mortgage rate stacks and because these queries include requirements that are not well suited to rigid filters or algorithms that seek exact matches. In addition, traditional AI-based approaches can overwhelm conventional reasoning models with complete sets of data, for example, mortgage product records that can number in the hundreds of thousands. That is, presenting a large language model with so much information that it is unable to hold attention/context when performing such a complex task. SUMMARY OF INVENTION Embodiments described herein overcome the above-described problems found with conventional computer-based mortgage systems. For example, various embodiments provide an agent-based AI system including intent classification. An intent classifier includes a large language model that operates to classify an intent of a user query and identify an action handler to orchestrate operation of the system using a predefined workflow. The multi-agent system includes a plurality of agents each including an associated large language model. The individual large language models can be individually fine-tuned with examples to optimize the respective agent for the reasoning task(s) to which it is dedicated. The action handlers designated for specified actions provide the logic for the defined workflow to complete the respective action using the multi-agent system. The structure of the overall agentic framework provides a dynamic toolset that allows a query to be resolved through a series of actions performed by specialized agents assigned to a task by the action handler. This structure provides more consistent operation by reducing the complexity of the individual task(s) or sub-task(s) performed by each agent. The system also provides a discrete focus that provides more consistent results because the action handler has a predefined set of tasks for completion by a predefined set of agents in a predefined order to resolve a user's query. This resolves queries using large language models to deliver deterministic results. The agent-based AI system also includes a generalist agent that is employed to respond to user messages for which a specific intent is not identified by the intent classifier. This provides the system with an ability to respond in context to messages for which only a general intent is identified by the intent classifier. The use of a conversational agent or chat widget provides the system with an ability for an on-going dialog with the user that can result in an identification of a specific intent, in a form of “self-heali