US-12619833-B2 - Digital processing systems and methods for implementing and managing artificial intelligence functionalities in applications

Abstract

Systems and methods are disclosed for selection operations for improving quality of Artificial Intelligence responses. The operations include accessing an application that employs AI functionality, receiving from a user, via the application, a query for which a response is sought from an AI agent, analyzing the query for determining a context, based on the context, selecting a particular AI agent from a pool of a plurality of AI agents, to which the query should be sent for response, and directing the query to the selected AI agent.

Inventors

• Nadav Grinberg
• Etay Liberman
• Vlad MYSTETSKYI

Assignees

• Monday.com Ltd.

Dates

Publication Date

20260505

Application Date

20231229

Claims (20)

1. 1 . A non-transitory computer-readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform selection operations for improving quality of Artificial Intelligence responses, the operations comprising: accessing an application that employs AI functionality, the application including a user interface and a plurality of platform elements, wherein each of the plurality of platform elements is configurable to communicate with at least one of a plurality of AI agents; receiving from a user, via at least one of the plurality of platform elements, a query for which a response is sought from an AI agent of the plurality of AI agents; analyzing the query for determining a context; based on the context, selecting a particular AI agent from a pool of the plurality of AI agents, to which the query should be sent for response, by accessing a repository associating a plurality of contexts with differing AI agents in the pool; directing the query to the selected AI agent; receiving a response from the selected AI agent; modifying, by the application, the received response; and conveying the modified response to the user via the at least one of the plurality of platform elements.
2. 2 . The non-transitory computer-readable medium of claim 1 , wherein the operations further include receiving user feedback on the conveyed response.
3. 3 . The non-transitory computer-readable medium of claim 2 , wherein the operations further include calculating a score based on the received user feedback.
4. 4 . The non-transitory computer-readable medium of claim 3 , wherein the operations further include storing, in a data structure, the query, the response of the selected AI agent, the user feedback and the calculated score.
5. 5 . The non-transitory computer-readable medium of claim 3 , wherein the operations further include using the calculated score for a future selection of an AI agent from the pool of a plurality of AI agents.
6. 6 . The non-transitory computer-readable medium of claim 3 , wherein the operations further include, when the calculated score is below a predetermined threshold, selecting an alternative AI agent from the pool of a plurality of AI agents, and transmitting the query to the alternative AI agent.
7. 7 . The non-transitory computer-readable medium of claim 3 , wherein the operations further include running a compliance check on the selected AI agent, using a compliance check plug-in-tool.
8. 8 . The non-transitory computer-readable medium of claim 7 , wherein calculating the score include calculating the score based on the received user feedback and the run compliance check.
9. 9 . The non-transitory computer-readable medium of claim 3 , wherein the calculated score is used to amend a predetermined prompt.
10. 10 . The non-transitory computer-readable medium of claim 1 , wherein selecting the particular AI agent from the pool of a plurality of AI agents includes accessing historical data on an ability of the particular AI agent to provide satisfactory responses in a context similar to the determined context.
11. 11 . The non-transitory computer-readable medium of claim 1 , wherein the operations further include accessing a library associating a plurality of queries with the pool of a plurality of AI agents.
12. 12 . The non-transitory computer-readable medium of claim 11 , wherein selecting the particular AI agent from the pool of a plurality of AI agents, includes selecting from the library the particular AI agent associated with one of the plurality of queries matching the received query.
13. 13 . The non-transitory computer-readable medium of claim 1 , wherein analyzing the query for determining a context includes at least one of: analyzing and breaking down a language used in the query; identifying keywords; classify the query; inferring an intent; analyzing details associated with the user, or a combination thereof.
14. 14 . The non-transitory computer-readable medium of claim 13 , wherein details associated with the user include at least one of a user profile, previous queries of the user, an ongoing conversation, a user location, or a combination thereof.
15. 15 . The non-transitory computer-readable medium of claim 1 , wherein the selected AI agent is an hierarchical agent.
16. 16 . The non-transitory computer-readable medium of claim 1 , wherein the operations further comprise selecting, based on the context, a second particular AI agent from the pool of the plurality of AI agents, to which the query should be sent for response and sending the query to the second particular AI agent.
17. 17 . The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise receiving a second response from the second particular AI agent, and modifying, by the application, the received second response.
18. 18 . The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise merging the modified response and the modified second response to produce a merged response.
19. 19 . The non-transitory computer-readable medium of claim 1 , wherein the plurality of distinct AI agents are operated by one or more other processors, and the at least one processor is communicatively coupled with the one or more other processors via one or more networks.
20. 20 . The non-transitory computer-readable medium of claim 1 , wherein the context is based in part on information associated with the at least one of the plurality of platform elements, and wherein the context is stored for later use by the application or by the plurality of AI agents.

Description

PRIORITY This application is a continuation of International Application No. PCT/IB2023/061992, filed on Nov. 28, 2023, which claims the benefit of U.S. Provisional Application No. 63/519,519, filed on Aug. 14, 2023, both of which are incorporated herein in their entirety. TECHNICAL FIELD The present disclosure relates generally to Artificial Intelligence (AI) functionality implementation and management methods and digital systems. Consistent with the disclosed embodiments, non-transitory computer-readable storage media may store program instructions, which may be executable by at least one processing device and perform any of the steps and/or methods described herein. BACKGROUND Operation of modern enterprises can be complicated and time-consuming. In many cases, managing the operation of a single project requires integration of several employees, departments, and other resources of the entity. To manage the challenging operation, project management platforms may be used. Such platforms allow a user to organize, plan, and manage resources in collaboration with other users by providing a collaborative platform in which users share project-related information in order to optimize the time and resources spent on each project. Project management platforms have a primary goal of automating repetitive tasks, enhancing decision-making processes, and maximizing project efficiency to streamline all aspects of project management. To achieve this objective, these platforms harness the potential of Artificial Intelligence (AI) functionality by integrating and utilizing AI capabilities within their applications. However, implementing and managing AI capabilities within an application can be challenging due to a range of factors. Firstly, the integration of AI capabilities into existing applications or systems may involve different existing systems and/or disparate data sources. Ensuring smooth integration, scalability and compatibility with different software environments can pose challenges during the implementation process. In addition, there are many AI algorithms available, each with its own strengths and weaknesses and suitable for specific tasks. Selecting the most appropriate algorithm for a particular application requires expertise and experimentation. Tuning the selected algorithms to optimize performance can be a complex and time-consuming process. In addition, AI applications often need to comply with ethical and legal guidelines. In such situations, ensuring fairness, avoiding prejudice, and protecting users' privacy are factors to be managed. Addressing these considerations, such as data anonymization and transparency in algorithmic decision-making, can be a challenge when developing and deploying AI-based applications. The present disclosure describes solutions to address or overcome one or more of the above challenges, among other drawbacks of existing project management workflow systems employing AI functionality. SUMMARY Embodiments consistent with the present disclosure provide digital systems and methods for implementing and managing artificial intelligence functionalities in applications. The disclosed embodiments may be implemented using a combination of conventional hardware and software as well as specialized hardware and software. Some embodiments consistent with the present disclosure involve systems, methods and computer readable media for building an application incorporating AI functionality. Exemplary operations may include enabling access to a developer application framework associated with a SaaS platform, receiving application code for generating an application in the SaaS platform, and receiving a selection of an AI assistant add-on provided by the SaaS platform. The operations may further include establishing a link between the selected AI assistant add-on and at least one of a plurality of SaaS platform elements, enabling implementation of permissions for providing access to data from the at least one of the plurality of linked SaaS platform elements, and based on the implemented permissions, configuring a transfer of at least one of structured data or unstructured data from the at least one of the plurality of linked SaaS platform elements. Operations may further involve receiving AI integration code including at least one reference to the transferred at least one of structured data or unstructured data and at least one call to an AI agent, and publishing the application for selective use with at least one of the SaaS platform elements, wherein in use, the published application is configured to exchange data with the at least one of the plurality of SaaS platform elements. Some embodiments consistent with the present disclosure involve systems, methods, and computer readable media for performing selection operations for a plurality of distinct Artificial Intelligence (AI) agents. Exemplary operations include accessing an application that employs AI functionality, sending, via the app