EP-4736058-A1 - DIRECTIVE GENERATIVE THREAD-BASED USER ASSISTANCE SYSTEM

EP4736058A1EP 4736058 A1EP4736058 A1EP 4736058A1EP-4736058-A1

Abstract

Embodiments of the disclosed technologies include generating a first thread classification prompt based on a first thread portion of an online dialog involving a user of a computing device, sending the first thread classification prompt to a first large language model, receiving a first thread classification generated and output by the first large language model based on the first thread classification prompt, formulating a plan execution prompt based on the first thread classification, sending the plan execution prompt to a second large language model, receiving a second thread portion generated and output by the second large language model based on the plan execution prompt and the online dialog, and generating a label for a third thread portion of the online dialog.

Inventors

AMATRIAIN-RUBIO, Xavier
NARANG, Winnie
TU, YIYUAN
MUNOZ ALCALDE, Jaime
PASUMARTHY, NITIN
BACH, THAO
WILLIAMS, DAVID
GARIBA, PRIYANKA
BREMER, Christopher M.
LOPEZ, Carlos H.
MONESTIE, Pierre Y.
TECLEMARIAM, Laura
KASERA, Yamini
KAZI, Michaeel
FU, ZHOUTONG
WU, MUCHEN

Assignees

Microsoft Technology Licensing, LLC

Dates

Publication Date: 20260506
Application Date: 20240613

Claims (20)

1. A computer-implemented method comprising: generating (702) a first thread classification prompt based on a first thread portion of an online dialog involving a user of a computing device; sending(704) the first thread classification prompt to a first large language model; receiving (706) a first thread classification, wherein the first thread classification is generated and output by the first large language model based on the first thread classification prompt; formulating (708) a plan execution prompt based on the first thread classification, wherein in response to determining that at least one stored thread involving the user matches the first thread classification, the plan execution prompt is formulated based on the first thread portion and the at least one stored thread that matches the first thread classification; sending (710) the plan execution prompt to a second large language model; receiving (712) a second thread portion, wherein the second thread portion is generated and output by the second large language model based on the plan execution prompt and the online dialog; and generating (714) a label for a third thread portion of the online dialog, wherein the label is configured for display at the computing device, the label is based on the first thread classification, and the third thread portion comprises the first thread portion and the second thread portion.
2. The computer-implemented method of claim 1, further comprising: labeling a fourth thread portion of the online dialog based on a second thread classification generated and output by the first large language model, wherein the online dialog comprises a plurality of natural language threads.
3. The computer-implemented method of claim 1, wherein generating the first thread classification prompt comprises: sending the first thread portion to the first large language model; and receiving a tagged version of the first thread portion, wherein the tagged version of the first thread portion comprises entity data associated with the first thread portion and the entity data is generated and output by the first large language model based on data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
4. The computer-implemented method of claim 3, wherein generating the first thread classification prompt comprises: based on the tagged version of the first thread portion, retrieving a stored classification template, wherein the retrieved classification template comprises at least one instruction to be executed by the first large language model; and including the retrieved classification template and the retrieved data in the first thread classification prompt.
5. The computer-implemented method of claim 1, wherein generating the first thread classification prompt comprises: sending the online dialog to the first large language model; and receiving a threaded version of the online dialog, wherein the threaded version of the online dialog comprises the first thread portion and the threaded version is generated and output by the first large language model.
6. The computer-implemented method of claim 1, wherein formulating the plan execution prompt comprises: based on the first thread classification, retrieving data associated with the user from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion; and including the retrieved data in the plan execution prompt.
7. The computer-implemented method of claim 6. wherein formulating the plan execution prompt comprises: based on the first thread classification, retrieving a stored plan template, wherein the retrieved stored plan template comprises a plurality of instructions to be executed by the second large language model; and including the retrieved plan template and the retrieved data in the plan execution prompt.
8. The computer-implemented method of claim 1, wherein the second thread portion comprises a plurality of tasks selected, prioritized, and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
9. The computer-implemented method of claim 1, wherein the second thread portion comprises an assessment of a job that is summarized and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
10. The computer-implemented method of claim 1, wherein the second thread portion comprises a recommendation that is generated and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
11. A system comprising: at least one processor; and at least one memory' device coupled to the at least one processor, wherein the at least one memory device comprises instructions that, when executed by the at least one processor, cause the at least one processor to perform at least one operation comprising: generating (702) a first thread classification prompt based on a first thread portion of an online dialog involving a user of a computing device; sending (704) the first thread classification prompt to a first large language model; receiving (706) a first thread classification, wherein the first thread classification is generated and output by' the first large language model based on the first thread classification prompt; formulating (708) a plan execution prompt based on the first thread classification, wherein in response to determining that at least one stored thread involving the user matches the first thread classification, the plan execution prompt is formulated based on the first thread portion and the at least one stored thread that matches the first thread classification; sending (710) the plan execution prompt to a second large language model; receiving (712) a second thread portion, wherein the second thread portion is generated and output by the second large language model based on the plan execution prompt and the online dialog; and generating (714) a label for a third thread portion of the online dialog, wherein the label is configured for display at the computing device, the label is based on the first thread classification, and the third thread portion comprises the first thread portion and the second thread portion.
12. The system of claim 11, wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising: sending the first thread portion to the first large language model; receiving a tagged version of the first thread portion, wherein the tagged version of the first thread portion comprises entity data associated with the first thread portion and the entity data is generated and output by the first large language model based on data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion; based on the tagged version of the first thread portion, retrieving a stored classification template, wherein the retrieved classification template comprises at least one instruction to be executed by the first large language model; and including the retrieved classification template and the retrieved data in the first thread classification prompt.
13. The system of claim 11, wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising: based on the first thread classification, retrieving data associated with the user from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion; including the retrieved data in the plan execution prompt; based on the first thread classification, retrieving a stored plan template, wherein the retrieved stored plan template comprises a plurality of instructions to be executed by the second large language model; and including the retrieved plan template and the retrieved data in the plan execution prompt.
14. The system of claim 11, wherein the second thread portion comprises a plurality of tasks selected, prioritized, and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
15. The system of claim 11, wherein the processor detects increases or decreases in latency of outputting the second thread portion and in response to detecting an increase in latency: reduces a number of the input signals, or uses a large language model with a reduced size as compared with the second large language model, or reduces a size of the second thread portion. .
16. The system of claim 11, wherein the second thread portion comprises: either a recommendation that is generated and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion; or an assessment of a job that is summarized and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
17. At least one non-transitory machine readable storage medium comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform at least one operation comprising: generating (702) a first thread classification prompt based on a first thread portion of an online dialog involving a user of a computing device: sending (704) the first thread classification prompt to a first large language model; receiving (706) a first thread classification, wherein the first thread classification is generated and output by the first large language model based on the first thread classification prompt; formulating (708) a plan execution prompt based on the first thread classification, wherein in response to determining that at least one stored thread involving the user matches the first thread classification, the plan execution prompt is formulated based on the first thread portion and the at least one stored thread that matches the first thread classification; sending (710) the plan execution prompt to a second large language model; receiving (712) a second thread portion, wherein the second thread portion is generated and output by the second large language model based on the plan execution prompt and the online dialog; and generating (714) a label for a third thread portion of the online dialog, wherein the label is configured for display at the computing device, the label is based on the first thread classification, and the third thread portion comprises the first thread portion and the second thread portion.
18. The at least one non-transitory machine readable storage medium of claim 17, wherein the second thread portion comprises a plurality of tasks selected, prioritized, and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
19. The at least one non-transitory machine readable storage medium of claim 17, wherein the second thread portion comprises an assessment of a job that is summarized and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.
0. The at least one non-transitory machine readable storage medium of claim 17, wherein the second thread portion comprises a recommendation that is generated and output by the second large language model based on the plan execution prompt, the online dialog, and data associated with the user retrieved from at least one of a stored thread, a data source, an entity connection graph, a domain application, or a recommendation system in response to receipt of the first thread portion.

Description

DIRECTIVE GENERATIVE THREAD-BASED USER ASSISTANCE SYSTEM TECHNICAL FIELD 1001] A technical field to which the present disclosure relates includes computer programs that use artificial intelligence to understand user requests for assistance and automate responses to those requests in a manner that simulates human conversation. Another technical field to which the present disclosure relates is generative artificial intelligence. COPYRIGHT NOTICE [002] This patent document, including the accompanying drawings, contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction of this patent document, as it appears in the publicly accessible records of the United States Patent and Trademark Office, consistent with the fair use principles of the United States copyright laws, but otherwise reserves all copyright rights whatsoever. BACKGROUND [003] A search engine is a software system that is designed to find and retrieve stored information that matches a search query. A chatbot (or chat bot) is a soft are application that can retrieve information and answer questions by simulating a natural language conversation with a human user. BRIEF DESCRIPTION OF THE DRAWINGS [004] The disclosure will be understood more fully from the detailed description given below and from the accompanying drawings of various embodiments of the disclosure. The drawings are for explanation and understanding only and should not be taken to limit the disclosure to the specific embodiments shown. [005] FIG. 1A is a flow diagram of an example method for directive generative thread-based user assistance using components of a directive generative thread-based user assistance system in accordance with some embodiments of the present disclosure. [006] FIG. IB is a flow diagram of an example method for generating a thread classification prompt using components of a directive generative thread-based user assistance system in accordance with some embodiments of the present disclosure. [007] FIG. 1C is a flow diagram of an example method for generating a plan execution prompt using components of a directive generative thread-based user assistance system in accordance with some embodiments of the present disclosure. [008] FIG. ID is a block diagram of an example architecture for a computing system in accordance with some embodiments of the present disclosure. [009] FIG. 2A is a timing diagram showing an example of communications between a thread- based user assistance interface and components of a directive generative thread-based user assistance system in accordance with some embodiments of the present disclosure. [0010] FIG. 2B is a timing diagram showing an example of using multiple threads to generate plan execution prompts in accordance with some embodiments of the present disclosure. [0011] FIG. 2C is a flow diagram showing an example of contextual content generation by a generative model in accordance with some embodiments of the present disclosure. [0012] FIG. 3A, FIG. 3B, FIG. 3C, FIG. 3D, FIG. 3E, FIG. 3F, FIG. 3G, FIG. 3H, FIG. 31, FIG. 3J, FIG. 3K, FIG. 3L, FIG. 3M, FIG. 3N, FIG. 30, FIG. 3P, FIG. 3Q, FIG. 3R, FIG. 3S. FIG. 3T. FIG. 3U, and FIG. 3V illustrate an example of at least one flow including screen captures of user interface screens configured to provide directive generative thread-based user assistance in accordance with some embodiments of the present disclosure. [0013] FIG. 4A and FIG. 4B illustrate an example of at least one flow including screen captures of user interface screens configured to provide directive generative thread-based user assistance in accordance with some embodiments of the present disclosure. [0014] FIG. 5 is a block diagram of a computing system that includes a directive generative thread-based user assistance system in accordance with some embodiments of the present disclosure. [0015] FIG. 6 is an example of an entity graph in accordance with some embodiments of the present disclosure. [0016] FIG. 7 is a flow diagram of an example method for directive generative thread-based user assistance using components of a directive generative thread-based user assistance in system in accordance with some embodiments of the present disclosure. [0017] FIG. 8 is a block diagram of an example computer system including components of a directive generative thread-based user assistance system in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION [0018] People often turn to their computing devices when they need an answer to a question or another type of assistance, such as help with a job search. Conventional search engines require the user to explicitly provide or select search terms that identify the kind of information the user is looking for. While search engines are ubiquitous, it remains an ongoing challenge to design a search engine to accurately interpret user queries on an individualized basis because every user has a uniqu