US-20260127383-A1 - SEARCH AND ANSWER GENERATION ENGINE FOR DATA SUMMARIZATION FROM MULTIPLE DATA SOURCES

US20260127383A1US 20260127383 A1US20260127383 A1US 20260127383A1US-20260127383-A1

Abstract

There are provided systems and methods for a search and answer generation engine for data summarization from multiple data sources. An online transaction processor or other service provider may provide computing services and platforms to entities, which may include live agent and self-service assistance features for answering users'questions. To provide more comprehensive searching and automated answer generation, the service provider may utilize an answer engine that may search multiple data sources in different data formats. Keywords may be extracted from a natural language question using an embedding LLM, and API calls to search features of each data source may be executed to retrieve relevant content. A summarization LLM may then concisely summarize the different content in different formats so that an answer may be provided. The user may then refine their question with further questions or requests, which may adjust the keywords and/or summarization.

Inventors

Siyuan Feng
Wei Yuan
Ciaran Maceochaidh

Assignees

PAYPAL, INC.

Dates

Publication Date: 20260507
Application Date: 20241106

Claims (20)

1 . A method comprising: receiving, via a user interface (UI) of an application, a question based on content from a plurality of distinct data sources; determining one or more keywords in the question using an embedding large language model (LLM) of a generative artificial intelligence (AI) system, wherein the one or more keywords are determined based on a semantic analysis of embeddings generated from the question by the embedding LLM; performing a search of the content from the plurality of data sources using the one or more keywords, wherein the search is performed in a plurality of data formats for the plurality of data sources using application programming interface (API) calls to APIs associated with search functions of the plurality of distinct data sources; identifying one or more matches of the content from the plurality of distinct data sources to the question based on the search, wherein each of the one or more matches comprises data in a corresponding one of the plurality of data formats; generating an answer to the question based on the one or more matches using a summarization LLM of the generative AI system, wherein the answer comprises a text generated by the summarization LLM from the plurality of data formats; and outputting, via the UI, the answer to the question.
2 . The method of claim 1 , wherein the question comprises a natural language question, and wherein the determining the one or more keywords includes predicting an intent and determining a context for the question using a natural language processor (NLP) of the generative AI system.
3 . The method of claim 1 , wherein the identifying the one or more matches comprises: converting the question to a first vector using the embedding LLM; converting search results from the search to one or more second vectors using the embedding LLM; comparing the first vector to the one or more second vectors based on a vector comparison function; and determining the one or more matches from the search results based on the vector comparison function and a similarity threshold.
4 . The method of claim 3 , wherein the determining the one or more keywords comprises: extracting the one or more keywords from the question using an NLP and one or more third vectors generated for the one or more keywords by the embedding LLM, wherein the converting the question to the first vector is based, at least in part, on the one or more third vectors generated for the one or more keywords.
5 . The method of claim 1 , wherein the generating the answer comprises prompting the summarization LLM with an instruction to generate the answer using the one or more matches each in the corresponding one of the plurality of data formats, and wherein the answer summarizes the one or more matches in a text format corresponding to the question.
6 . The method of claim 1 , further comprising: receiving feedback associated with the answer; and updating at least one data retrieval module associated with the performing the search based on the feedback.
7 . The method of claim 6 , wherein the at least one data retrieval module comprises a retrieval augmented generation (RAG) module, and wherein the updating comprises at least one of: retraining the RAG module based on the feedback; or updating one or more data source retrieval criteria of the RAG module based on the feedback.
8 . The method of claim 1 , wherein the answer comprises a summarization of the one or more matches and the one or more matches ranked based on a relevancy score of each of the one or more matches to the question, and wherein the summarization and the one or more matches ranked are provided via the UI for the answer.
9 . The method of claim 1 , wherein the plurality of data sources comprises at least one of internal content or internal resources of a service provider, and wherein the plurality of data sources include at least one of an internal chat platform, a service ticketing platform, a code collaboration workspace platform, or computing service documentation.
10 . A system comprising: a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to execute instructions to cause the system to: determine a set of keywords for a question using an embedding large language model (LLM) of a generative artificial intelligence (AI) system, wherein the set of keywords is determined based on a semantic analysis of embeddings generated from the question by the embedding LLM; perform a search of a plurality of data sources using at least one application programming interface (API) call to an API of search of the plurality of data sources, wherein the API is configured to search a corresponding one of the plurality of data sources for content associated with the set of keywords; identify the content from the plurality of data sources based on the search, wherein the content comprises data in one of a plurality of data formats for the corresponding one of the plurality of data sources; generate an answer to the question based on the content using a summarization LLM of the generative AI system, wherein the answer comprises a text generated by the summarization LLM from the plurality of data formats; and output the answer to the question.
11 . The system of claim 10 , wherein the answer further comprises one or more links in the text to the content from the plurality of data sources.
12 . The system of claim 11 , wherein the one or more links are associated with one or more citations in corresponding portions of the text to the content.
13 . The system of claim 10 , wherein the question comprises a natural language question, and wherein the determining the set of keywords includes predicting an intent and determining a context for the question using a natural language processor (NLP) of the generative AI system.
14 . The system of claim 10 , wherein the at least one API call to the API utilizes a search function associated with the API and the one of the data formats to search the corresponding one of the plurality of data sources.
15 . The system of claim 10 , wherein executing the instructions further causes the system to: receive an additional question that requests one of a refinement of the answer or additional information associated with the content used for the text in the answer, wherein the additional question is associated with the question previously asked; determine a change to the content based on the additional question; and update the answer using the summarization LLM and based on the change to the content.
16 . The system of claim 15 , wherein the additional question is received via a user interface field provided with the answer for the refinement or the additional information.
17 . A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising: receiving a set of keywords for a question asked by a user via a user interface (UI); executing a search for one or more matches of content from the plurality of data sources using the set of keywords, wherein the executing the search comprises calling an application programming interface (API) of one of a search engine or a search function associated with each of the plurality of data sources with a request to search a corresponding one of a plurality of data sources based on the set of keywords; based on search results from executing the search, generating an answer to the question using a summarization large language model (LLM) of a generative artificial intelligence (AI) system, wherein the answer comprises a text generated by the summarization LLM based on the search results; and providing the answer to the question via the UI.
18 . The non-transitory machine-readable medium of claim 17 , wherein, prior to the receiving the set of keywords, the operations further comprise: generating the set of keywords from the question using an embedding LLM of the generative AI system, wherein the set of keywords are determined based on a semantic analysis of embeddings generated from the question by the embedding LLM.
19 . The non-transitory machine-readable medium of claim 17 , wherein the answer is requested to be provided in natural language as a summarization of the content in place of search results via the UI.
20 . The non-transitory machine-readable medium of claim 17 , wherein the plurality of data sources are associated with at least one of an internal chat platform, a service ticketing platform, a code collaboration workspace platform, or computing service documentation.

Description

TECHNICAL FIELD The present disclosure relates generally to generative artificial intelligence (AI) and models, and more specifically to cross-domain and data source search and answer generation engines using large language models (LLMs) and other generative AIs. BACKGROUND Online service providers may offer various services to end users, merchants, and other entities. This may include providing computing services through different software applications, websites, platforms, and resources, such as those that may be involved with digital transaction processing. Further, the service provider may provide and/or facilitate the use of applications and websites for online payments, peer-to-peer (P2P) transfers, and/or other computing services to different entities including merchants or other entities and their corresponding users (e.g., code developers, employees, agents, etc.). However, use of these computing services may require implementation by new and foreign systems, which may require specific assistance. Users may encounter difficulties in finding the required resources and instructions, and personalized assistance or human agents is costly and may not be widely available to assist these entities. Further, data sources that may assist entities may be distributed across many different domains, platforms, and sources. Thus, it is desirable to automate labor-intensive processes for efficiently providing accurate answers to questions by users of the entities, and there is a need for an automated, intelligent, and efficient computing system and framework for search and answer generation across different domains, data formats, and sources. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a networked system suitable for implementing the processes described herein, according to an embodiment; FIG. 2 is an exemplary system environment where a client device may make a request corresponding to a question to be answered by an intelligent answer generation engine of a service provider, according to an embodiment; FIGS. 3A and 3B are exemplary user interfaces of an intelligent search and answer generation engine for multiple data sources, according to various embodiments; FIG. 4 is a flowchart of an exemplary process for a search and answer generation engine for data summarization from multiple data sources, according to an embodiment; and FIG. 5 is a block diagram of a computer system suitable for implementing one or more components in FIG. 1, according to an embodiment. Embodiments of the present disclosure and their advantages are best understood by referring to the detailed description that follows. It should be appreciated that like reference numerals are used to identify like elements illustrated in one or more of the figures, wherein showings therein are for purposes of illustrating embodiments of the present disclosure and not for purposes of limiting the same. DETAILED DESCRIPTION Provided are methods for a search and answer generation engine for data summarization from multiple data sources. Systems suitable for practicing methods of the present disclosure are also provided. A service provider, such as an online transaction processor, may provide computing services to users and/or their corresponding entities, which may include individual customers or other individuals, merchant customers of an online transaction processor, businesses and their representatives and/or employees, and the like. These computing services may include those associated with electronic transaction processing, P2P payments and transfers, cryptocurrency trading, and other computing services involved with payment processing. For these computing services, merchants may desire to utilize the services and/or incorporate the services with their computing platforms, while individual users may encounter issues that require assistance or instruction. This may require performance of specific tasks and operations, and therefore users may have questions and inquiries requiring assistance. Conventionally, this type of assistance and instruction is provided through static data provided through instructional materials and/or other available or searchable information. The service provider may also utilize live agents and/or chatbots to provide responsive assistance to user outreach; however, these resources are limited in scope and/or availability. For example, frontline support may spend a significant time searching for relevant information to triage, investigate, and respond to issues across various platforms of a service provider. For example, a service provider may include internal platforms, systems, and/or applications that each have corresponding content and data, such as coding platforms and repositories, messaging and communication applications, historical cases, ticketing software, support tickets, and developer documents. While handling numerous cases, it is time-consuming and challenging for human agents to handle analytical