US-20260127382-A1 - LARGE LANGUAGE MODEL HALLUCINATION ATTENUATION BASED ON MULTI-AGENT DEBATE

Abstract

A action-flow including one or more proposed actions derived from the response of the least one agent with the adopted personality engaged in the debate, within the personality council is determined. The selection of at least one agent and at least one critic from a group of agents and critics, based on a user's query, is performed. The construction of a personality council for the large language model (LLM) based on the user's query and at least one agent and at least one critic is performed. The conduction one or more rounds of a debate, within the personality council, among the at least one agent and at the least one critic from the group of agents and critics, wherein a response of the least one agent with the adopted personality is critiqued by the least one critic, based on the user's query, until a threshold for concluding the debate is met or exceeded, is also performed.

Inventors

• Zhong Fang Yuan
• Wen Wang
• Tong Liu
• He Li
• Haechul Shin

Assignees

• INTERNATIONAL BUSINESS MACHINES CORPORATION

Dates

Publication Date

20260507

Application Date

20241104

Claims (20)

1. 1 . A computer-implemented method for attenuating hallucinations in a large language model (LLM), the method comprising: selecting at least one agent and at least one critic from a plurality of agents and critics, based on a user's query; constructing a personality council for the large language model (LLM) based on the user's query and the at least one agent and at the least one critic from the plurality of agents and critics, wherein the personality council instructs the least one agent to adopt a personality; conducting one or more rounds of a debate, within the personality council, among the at least one agent and at the least one critic from the plurality of agents and critics, wherein a response of the least one agent with the adopted personality is critiqued by the least one critic, based on the user's query, until a threshold for concluding the debate is met or exceeded; and building an action-flow comprised of one or more proposed actions derived from the response of the least one agent with the adopted personality engaged in the debate, within the personality council.
2. 2 . The method of claim 1 , further comprising: displaying an answer based the action-flow.
3. 3 . The method of claim 2 , wherein: the selecting at least one agent includes selecting three agents and the personality council instructs the three agents to adopt distinct personalities.
4. 4 . The method of claim 3 , wherein: the constructing the personality council for the large language model (LLM) is further based on a selection of at least one decision maker.
5. 5 . The method of claim 4 , wherein: constructing the personality council for the large language model (LLM) further comprises: the personality council instructing the decision maker to moderate a debate.
6. 6 . The method of claim 5 , wherein: the conducting the one or more rounds of the debate, within the personality council further comprises: concluding a round of a debate when a second threshold is met or exceeded.
7. 7 . The method of claim 3 , wherein: the conducting the one or more rounds of the debate, within the personality council further comprises: concluding a round of the debate when a second threshold is met or exceeded.
8. 8 . A computer usable program product comprising one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by a processor to cause the processor to perform operations attenuating hallucinations in a large language model (LLM) comprising: selecting at least one agent and at least one critic from a plurality of agents and critics, based on a user's query; constructing a personality council for the large language model (LLM) based on the user's query and the at least one agent and at the least one critic from the plurality of agents and critics, wherein the personality council instructs the least one agent to adopt a personality; conducting one or more rounds of a debate, within the personality council, among the at least one agent and at the least one critic from the plurality of agents and critics, wherein a response of the least one agent with the adopted personality is critiqued by the least one critic, based on the user's query, until a threshold for concluding the debate is met or exceeded; and building an action-flow comprised of one or more proposed actions derived from the response of the least one agent with the adopted personality engaged in the debate, within the personality council.
9. 9 . The computer usable program product of claim 8 , further comprising: displaying an answer based the action-flow.
10. 10 . The computer usable program product of claim 9 , wherein: the selecting at least one agent includes selecting three agents and the personality council instructs the three agents to adopt distinct personalities.
11. 11 . The computer usable program product of 10 , wherein: the constructing the personality council for the large language model (LLM) is further based on a selection of at least one decision maker.
12. 12 . The computer usable program product of 11 , wherein: constructing the personality council for the large language model (LLM) further comprises: the personality council instructing the decision maker to moderate a debate.
13. 13 . The computer usable program product of claim 12 , wherein: the conducting the one or more rounds of the debate, within the personality council further comprises: concluding a round of a debate when a second threshold is met or exceeded.
14. 14 . The computer usable program product of claim 10 , wherein: the conducting the one or more rounds of the debate, within the personality council further comprises: concluding a round of the debate when a second threshold is met or exceeded.
15. 15 . A computer system comprising a processor and one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by the processor to cause the processor to perform operations attenuating hallucinations in a large language model (LLM) comprising: selecting at least one agent and at least one critic from a plurality of agents and critics, based on a user's query; constructing a personality council for the large language model (LLM) based on the user's query and the at least one agent and at the least one critic from the plurality of agents and critics, wherein the personality council instructs the least one agent to adopt a personality; conducting one or more rounds of a debate, within the personality council, among the at least one agent and at the least one critic from the plurality of agents and critics, wherein a response of the least one agent with the adopted personality is critiqued by the least one critic, based on the user's query, until a threshold for concluding the debate is met or exceeded; and building an action-flow comprised of one or more proposed actions derived from the response of the least one agent with the adopted personality engaged in the debate, within the personality council.
16. 16 . The computer system of claim 15 , further comprising: displaying an answer based the action-flow.
17. 17 . The computer system of claim 16 , wherein: the selecting at least one agent includes selecting three agents and the personality council instructs the three agents to adopt distinct personalities.
18. 18 . The computer system of claim 17 , wherein: the constructing the personality council for the large language model (LLM) is further based on a selection of at least one decision maker.
19. 19 . The computer system of claim 18 , wherein: constructing the personality council for the large language model (LLM) further comprises: the personality council instructing the decision maker to moderate a debate.
20. 20 . The computer system of claim 19 , wherein: the conducting the one or more rounds of the debate, within the personality council further comprises: concluding a round of a debate when a second threshold is met or exceeded.

Description

BACKGROUND The present invention relates generally to question-answer generation using large language models (LLMs). More particularly, the present invention relates to a method, system, and computer program designed for attenuating hallucinations in large language models (LLM) using an agent-based perspectives, within a debate/critic-actor framework. One challenge, recognized by the illustrative embodiments of the invention, is that businesses must carefully consider managing hallucinations if or when they plan to deploy large language models (LLMs) in business-critical applications. Businesses generally require such business-critical applications to be reliable and robust, but large language models (LLMs) may experience hallucinations, especially when large language models (LLMs) generating complex content, such as content related to ambiguous subject matter. The embodiments of the invention further acknowledge that even when large language models (LLMs) use agents and retrieval-augmented generation (RAG), a method that theoretically improves accuracy by combining the retrieval of relevant external documents with the model's generative capabilities, hallucinations can still occur at unacceptable frequencies. For example, hallucinations can result in incorrect answers or inaccurate JavaScript Object Notation (JSON) outputs. These hallucinations may deter businesses running mission-critical applications from adopting LLMs in their infrastructure, where stable performance and interpretability are highly valued. Additionally, as recognized by the illustrative embodiments of the invention, hallucinations may negatively affect user experiences and undermine customer trust in the mission-critical applications. When users or customers repeatedly encounter inconsistent or incorrect information, the overall confidence of the users or customers in the mission-critical application's reliability diminishes, potentially leading to customer dissatisfaction and loss of business. Ensuring that large language models (LLMs) generate reliable and accurate content and answers is useful to maintaining positive business reputations and achieving long-term success. For example, as recognized by the illustrative embodiments of the invention, when a large language model (LLM) provides incorrect answers or erroneous data in areas like customer support, financial analysis, or decision-making tools, these incorrect answers or erroneous data may lead to flawed conclusions and operational disruptions. Businesses may suffer losses in business efficiency and weakened trust from clients and stakeholders, and possibly face legal or compliance issues if the incorrect answers or erroneous outputs lead to regulatory non-compliance or contractual breaches. Therefore, the illustrative embodiments recognize that it would be desirable to have methods, systems, and computer programs designed for attenuating hallucinations in large language models (LLM) using an agent-based perspective within a debate/critic-actor framework. SUMMARY The illustrative embodiments provide for optimizing reflective method based on multi-personality driven LLM-agents. An embodiment includes selecting at least one agent and at least one critic from a plurality of agents and critics, based on a user's query. The embodiment includes constructing a personality council for the large language model (LLM) based on the user's query and at least one agent and at least one critic from the plurality of agents and critics. The embodiment includes conducting one or more rounds of a debate, within the personality council, among at least one agent and at least one critic from the plurality of agents and critics. The embodiment includes building an action-flow comprised of one or more proposed actions derived from the response of the least one agent with the adopted personality engaged in the debate, within the personality council. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the embodiment. An embodiment includes a computer usable program product. The computer usable program product includes a computer-readable storage medium, and program instructions stored on the storage medium. An embodiment includes a computer system. The computer system includes a processor, a computer-readable memory, and a computer-readable storage medium, and program instructions stored on the storage medium for execution by the processor via the memory. BRIEF DESCRIPTION OF THE DRAWINGS The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of the illustrative embodiments when read in conjunction with the accompanying drawings, wherei