Search

EP-4742097-A1 - COMPUTER-IMPLEMENTED METHOD AND DEVICE FOR ACCESSING DATA REGARDING A COMPLEX SYSTEM

EP4742097A1EP 4742097 A1EP4742097 A1EP 4742097A1EP-4742097-A1

Abstract

Modern chatbots and so-called co-pilots (software systems that performs a task in interaction with a user) can assist in a wide range of tasks, from explaining the workings of a machine to answering queries to providing troubleshooting recommendations, and even generating code for specialized industrial devices. However, the repetitive invocation of LLM requests for function evaluation and planning also poses challenges: regarding Time and Money: the cost of token generation by the LLM is quite high, repetitive invocation implies higher expenditures. The presented invention solves the problem mentioned by developing a strategy that focuses on two key aspects. • The first aspect involves the storage of previous experiences and observations in a way that it becomes a knowledge base, so much so, that conclusions can be drawn solely from the knowledge base without even contacting the LLM. • The second aspect is the implementation of an algorithm that utilizes the knowledge base and the LLM to fulfill user instructions and explore new actions. This dual-pronged approach ensures that the system is both reactive, in that it can respond to immediate user instructions based on past experiences, and proactive, in that it can explore new actions that may not have been explicitly instructed by the user. In the following sections, both aspects are discussed in detail, explaining how they contribute to the overall functionality of the solution.

Inventors

  • Awais, Muhammad Usman

Assignees

  • Siemens Aktiengesellschaft

Dates

Publication Date
20260513
Application Date
20241107

Claims (20)

  1. Computer-implemented method for accessing data regarding a complex system (SYS), the complex system comprising multiple interacting system elements (SE1, SE2, SEn), with the following steps: i) Receiving a query for fulfilling a task on the complex system (SYS), by a Knowledge Base (KB) j) Checking whether the same or a similar query has been processed by the Knowledge Base (KB) before (101) and if there is a plan consisting of preprocessed list of suitable actions on the request stored, send a response (11) with the suitable actions, k) Or else, if available, gather available knowledge about the complex system (SYS) and/or system elements (SE1, SE2, SEn), for achieving a response to the request, l) Find a list of suitable actions as a response to the request, (131), m) Choose the best succession of actions out of the list of suitable actions, and create a plan to fulfill the task, (16) n) Save the created plan in the Knowledge Base (KB), o) if the created plan fulfills the query, then send response to query, (172) p) or else return to step d).
  2. Computer-implemented method according to claim 1, characterized in that at least part of the data is generated by at least one of the elements in the complex system.
  3. Computer-implemented method according to claim 1 or 2, characterized in that at least part of the data consists of documents related to at least one of the system elements, in particular documents containing descriptions of the at least one system element.
  4. Computer-implemented method according to one of the preceding claims, characterized in that the Knowledge Base (KB) contains data that is collected and stored in in the form of a time series database, wherein there are stored relations regarding Actions (34), dependencies of Actions (35) and Transformations (32).
  5. Computer-implemented method according to one of the preceding claims, characterized in that for step b) the similarity of a query is calculated using a distance metric based on an embedding of the request.
  6. Computer-implemented method according to one of the preceding claims, characterized in that the request is formulated in a formal language.
  7. Computer-implemented method according to one of the preceding claims, characterized in that the available knowledge collected in step c) comprises information about - The sender of the request, - a history of interactions with the sender of the request, - The system or the interacting system elements, in particular technical documentation, - The domain, where the system is deployed.
  8. Computer-implemented method according to one of the preceding claims, characterized in that in step d) the finding a suitable action (A1, A2, ...), is performed by a Large Language Model (LLM) wherein the possible actions are grouped according to their distance from the request and only those actions (A5) are deemed suitable that are within a certain distance (R ) from the request.
  9. Computer-implemented method according to one of the preceding claims, characterized in that having one suitable action, it is searched in the Knowledge Base for an already stored plan containing the certain action, selecting the action that is in direct succession in the plan as a suitable next action that has been used in the stored plan.
  10. Computer-implemented method according to one of the preceding claims, characterized in that in the Knowledge Base (KB) also a feedback information is stored, on the degree of correspondence of the generated plan found on the request.
  11. Computer-implemented method according to one of the preceding claims, characterized in that an action has a certain type, and actions of the same type create a vector space.
  12. Computer-implemented method according to one of the preceding claims, characterized in that the complex system is a technical system, in particular a robot, and an action is manipulating one part of the technical system, in particular moving or administrating or maintenance.
  13. Computer Program Product suitable for executing the steps of the method according to the features of one of claims 1 to 12.
  14. Knowledge Base Device (KB) for accessing data regarding a complex system (SYS), the complex system (SYS) comprising multiple interacting system elements, (SE1, SE2, SEn), wherein the device offers the following functional units: q) Receiver for Receiving a query for fulfilling a task on the complex system, r) Processor for Calculating whether the same or a similar query has been processed by the Knowledge Base (KB) before (101) and if there is a plan consisting of preprocessed list of suitable actions on the request stored, Sender to send a response (11) with the suitable actions, s) Or else, if available, gathering available knowledge about the complex system (SYS) and/or system elements (SE1, SE2, SEn), for achieving a response to the request, t) Finding a list of suitable actions as a response to the request, (131), u) Choosing the best succession of actions out of the list of suitable actions, and create a plan to fulfill the task, (16) v) Memory for saving the created plan in the Knowledge Base (KB), w) Sender for sending a response to the query, if the created plan fulfills the query, (172).
  15. Device (KB) according to claim 14, characterized in that at least part of the data is generated by at least one of the elements in the complex system.
  16. Device (KB) according to claim 14 or 15, characterized in that at least part of the data consists of documents related to at least one of the system elements, in particular documents containing descriptions of the at least one system element.
  17. Device (KB) according to one of the preceding claims 14 to 16, characterized in that the Knowledge Base (KB) contains data that is collected and stored in in the form of a time series database, wherein there are stored relations regarding Actions (34), dependencies of Actions (35) and Transformations (32).
  18. Device (KB) according to one of the preceding claims 14 to 17, characterized in that the processor is able to calculate the similarity of a query using a distance metric based on an embedding of the request.
  19. Device (KB) according to one of the preceding claims 14 to 18, characterized in that the request is formulated in a formal language.
  20. Device (KB) according to one of the preceding claims 14 to 19, characterized in that the available knowledge collected comprises information about - The sender of the request, - a history of interactions with the sender of the request, - The system or the interacting system elements, in particular technical documentation, - The domain, where the system is deployed.

Description

Artificial Intelligence AI is a transformative technology with the potential to revolutionize industries. In the industrial sector, AI is driving increased efficiency, productivity, and innovation across areas like process optimization, predictive maintenance, and quality control. AI can analyze industrial data to identify patterns and opportunities for improvement, enabling fine-tuning of operations. Al-powered systems can predict equipment maintenance needs and automate quality inspections, improving reliability and reducing costs. As AI capabilities advance, its integration into industrial operations is expected to grow, boosting competitiveness and innovation. The recent advancements in Generative AI have significantly transformed the development of agents, chatbots, and co-pilots. The introduction of Large Language Models (LLM) models like the very popular GPT-3 and GPT-4, BERT or LLaMA have enabled these systems to generate human-like text based on the input they receive. This has resulted in more natural and engaging conversations, as these advanced models can understand the context and generate new content. Modern autonomous agents can now leverage the use of LLMs to interact with the environment in natural language. In recent times, companies that focuses on developing these Large Language Models (LLM) started partnering also with robotic companies to create robots that can interact with the environment, perform tasks using its robotic hands, and talk to a human being simultaneously. Figure 1 shows the situation as described as state of the art: on the left-hand side there is an application APP, 11, that might be used by a user (this could be a person or any technical system, depending on the use case). On the other side, there is the LLM, 12, which is questioned several times, 101, 102, ... 10n. The multiple disadvantages of this setting are explained below. Similarly, modern chatbots and so-called co-pilots (software systems that performs a task in interaction with a user) can now assist in a wide range of tasks, from explaining the workings of a machine to answering queries to providing troubleshooting recommendations, and even generating code for specialized industrial devices. However, the repetitive invocation of LLM requests for function evaluation and planning also poses challenges: Time: most LLMs take some time to generate tokens, which causes delays in preparing the response. In the case of autonomous agents, the delay in response from the LLM can cause a delay in planning the tasks, hence the response generation becomes even slower.Money: the cost of token generation by the LLM is quite high, repetitive invocation implies higher expenditures. Another problem with the interacting of agents, chatbots, and co-pilots that use LLMs is their very slow and costly learning process. This is mainly due to the cost of learning associated with LLMs. Here it is argued that to make an agent learn something it is not mandatory to make the LLM learn or fine-tune. Currently, there are two types of learning possible for an agent created using an LLM: In-context learning Here, the LLM learns and generates responses based on the given context. Unlike traditional machine learning models that require explicit labels for training, in-context learning leverages the surrounding text or data as implicit supervision. This allows the model to understand the nuances of the conversation or task at hand and generate more relevant and accurate responses. For instance, in a conversation, the model can refer to previous exchanges to maintain the flow and coherence of the dialogue. In-context learning is the basis for the Retrieval Augmented Generation (RAG). An LLM first retrieves relevant documents or information from a large database based on the given query or context. This retrieved information is then used as additional context for a generative model, which generates the final response. This approach allows the model to leverage the vast amount of information available in the database, while also maintaining the flexibility and creativity of generative models. The downside of in-context learning is that it is like short-term memory. In-context learning relies heavily on the immediate context. This means that the lessons learned during one session are not remembered for future sessions. The model starts each new conversation session without any knowledge of past sessions, which can limit its ability to build on previous learnings. If the previous conversation is saved, then parsing through all the history becomes a problem itself. Fine-tuning The other end of learning is the fine-tuning. Fine-tuning is a process in machine learning where a pre-trained model, such as an LLM, is further trained on a specific task using a smaller, task-specific dataset. This allows the model to adapt its general knowledge learned from the large pre-training dataset to the specific requirements of the task. In theory, the problem of for