Search

EP-4738167-A1 - RETRIEVAL-AUGMENTED GENERATIVE SYSTEM AND COMPUTER-IMPLEMENTED METHOD FOR PROTECTING CLASSIFIED INFORMATION IN A RETRIEVAL-AUGMENTED GENERATIVE SYSTEM

EP4738167A1EP 4738167 A1EP4738167 A1EP 4738167A1EP-4738167-A1

Abstract

Retrieval-augmented generative system comprising a user interface (21, 41) configured to receive a prompt (22, 42) from a user (28, 48), a retrieval unit (23, 43) configured to retrieve documents relevant to the prompt (22, 42), a concealment unit (24, 44) configured to identify at least one document among the retrieved documents comprising restricted information and to mark it being a confidential document, a large language model unit (25, 45) configured to input all retrieved documents including the at least one marked confidential document into a large language model and to output from the large language model a response containing restricted information and unrestricted information, and an output interface (27, 47) configured to provide the response (29, 49) comprising the restricted information in a concealed form to a user (28, 48).

Inventors

  • Büttner, Florian
  • Yang, Yinchong
  • STÖRMANN, Christof

Assignees

  • Siemens Aktiengesellschaft

Dates

Publication Date
20260506
Application Date
20241031

Claims (15)

  1. Retrieval-augmented generative system comprising - a user interface (21, 41) configured to receive a prompt (22, 42) from a user (28, 48), - a retrieval unit (23, 43) configured to retrieve documents relevant to the prompt (22, 42), - a concealment unit (24, 44) configured to identify at least one document among the retrieved documents comprising restricted information and to mark it being a confidential document, - a large language model unit (25, 45) configured to input all retrieved documents including the at least one marked confidential document into a large language model and to output from the large language model a response containing restricted information and unrestricted information, and - an output interface (27, 47) configured to provide the response (29, 49) comprising the restricted information in a concealed form to a user (28, 48).
  2. Retrieval-augmented generative system according to claim 1, wherein the concealment unit (24, 44) is configured to generate a concealed confidential document comprising the restricted information of the confidential document in a concealed form.
  3. Retrieval-augmented generative system according to any of the preceding claims, wherein the retrieved documents comprise information about an industrial environment (30, 50), preferably information for operating, monitoring or controlling a manufacturing process or manufacturing device.
  4. Retrieval-augmented generative system according to any of the preceding claims, wherein each of the retrieved documents is marked as confidential document depending on a confidentiality level of the user (28, 48).
  5. Retrieval-augmented generative system according to any of the preceding claims, wherein the concealment unit (24, 44) comprises a watermarking functionality applying a watermark to each of the confidential documents.
  6. Retrieval-augmented generative system according to claim 5, wherein the watermarking functionality reformulates any confidential document leaving the watermark that is parameterized with a seed.
  7. Retrieval-augmented generative system according to claim 5 or 6, wherein a postprocessing functionality (26, 46) is configured to perform a watermarking verification on the response using the seed to detect the watermarks in the response and redacting watermarked information with specific tokens disguising the watermarked information.
  8. Retrieval-augmented generative system according to any of claims 5 to 7, wherein the watermarking functionality and the postprocessing functionality (26, 46) are performed by a large language model.
  9. Retrieval-augmented generative system according to any of claims 1 to 4, wherein the concealment unit (24, 44) comprises a named entity recognition functionality, which identifies at least one pre-defined category of confidential entities in each of the confidential documents and encodes each of the confidential entities by a label indicating the identified category.
  10. Retrieval-augmented generative system according to claim 9, wherein the response (29, 49) is generated having labels instead of the confidential entities and the response is provided to the user interface.
  11. Retrieval-augmented generative system according to any of claim 9 and 10, wherein all confidential entities are encoded with an extended label indicating the category of the confidential entity and a classification level, which indicates the classification level of the confidential entity, wherein a mapping between each of the confidential entities and the extended label is stored in a table, wherein the response is decoded according to the table depending on the confidentiality level of the user (28, 48).
  12. Retrieval-augmented generative system according to claim 9 or 11, wherein the at least one pre-defined category is specific to the industrial environment (30, 50), preferably to the industrial process and/or industrial device.
  13. Computer-implemented method for protecting classified information in a retrieval-augmented generative system, comprising the steps - by a user interface, receiving (S1) a prompt from a user, - by a retrieval unit, retrieving (S2) documents relevant to the prompt, - by a concealment unit, identifying (S3) at least one document among the retrieved documents comprising restricted information and marking (S4) it being a confidential document, - by a large language model unit, inputting (S5) all retrieved documents including the at least one marked confidential document into a large language model and outputting (S6) from the large language model a response containing restricted information and unrestricted information, and - by an output interface, providing (S7) the response comprising the restricted information in a concealed form to a user.
  14. Computer-implemented method according to claim 13, comprising steps performed by a retrieval-augmented generative system according to claims 1 to 12.
  15. A computer program product directly loadable into the internal memory of a digital computer, comprising software code portions for performing the steps of claim 13 to 14 when said product is run on said digital computer.

Description

Field of the Invention The present disclosure relates to a retrieval-augmented generative system and a computer-implemented method protecting confidential information in a retrieval-augmented generative system. Background Generative artificial intelligence (generative Al) is artificial intelligence capable of generating text, images, videos, or other data using generative models in response to prompts. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics. Generative AI and in particular Large Language Models (LLMs) have demonstrated remarkable capabilities in creating relevant outputs to address user queries. However, current frameworks are rather complicated and opaque systems that are prone to produce unreliable output if users prompt the generative model to generate content that diverges from the training data. For industrial applications using generative AI for supporting monitoring, operation or maintenance of a machine, vehicle, building or power facilities, the output in response to a query needs to be reliable and explainable to get trust into the responses. A known approach to partially overcome such limitations is Retrieval-Augmented-Generation (RAG) where powerful generative models are combined with additional information obtained from a trustworthy knowledge base. Additionally, retrieval-augmented generative (RAG) systems are much more cost efficient since it circumvents the necessity of any fine-tuning steps. It is also a known issue that LLMs, especially the RAGs, may leak confidential information from retrieved documents. While an intuitive solution would be excluding certain documents from being retrieved results in an incomplete context for LLM, it often results in less qualitative responses due to "unknown unknown". In other words, the end user who reads the response of the LLM doesn't know what he/she doesn't know and may base his/her decision / reaction on such wrong conclusions. Therefore, it is an object of the present invention to improve the quality of the response without leaking confidential information. It is a further object of the present invention to provide a quality-optimized response depending on the confidentiality level of a user asking for a response. Brief Summary of the Invention This object is solved by the features of the independent claims. The dependent claims contain further embodiment of the invention. A first aspect concerns a Retrieval-augmented generative system comprising a user interface configured to receive a prompt from a user,a retrieval unit configured to retrieve documents relevant to the prompt,a concealment unit configured to identify at least one confidential document among the retrieved documents comprising restricted information and to mark it being confidential,a large language model unit configured to input all retrieved documents including the at least one marked confidential document into a large language model and to output from the large language model a response containing restricted information and unrestricted information, andan output interface configured to provide the response comprising the restricted information in a concealed form to a user. Advantageously, a user is now aware that further information is available to improve the content of the response. The provided response can be assessed with respect to further information which is available but not accessible by the user. This enhances the quality of, and information provided by the response. In an embodiment the concealment unit is configured to generate a concealed confidential document comprising the restricted information of the confidential document in a concealed form. This ensures that the RAG system is not able to leak restricted information in the provided response as the restricted information is not input into the large language model unit. The concealed document may contain unrestricted information in clear, i.e., comprehensive form, besides the concealed restricted information and enables the LLM unit to provide a response considering also the unrestricted information of the classified document. In an embodiment the retrieved documents comprise information about an industrial environment, preferably documents for operating, monitoring or controlling a manufacturing process or manufacturing device. These retrieved documents provide information, which is very specific to prompts, i.e., question when the RAG system is used in an industrial environment and thus "retrains" the LLM unit efficiently to provide an environment related and thus quality optimized response. In an embodiment each of the retrieved documents is marked as confidential document depending on a confidentiality level of the user. Advantageously, this allows the RAG system not only to identify all confidential document among the retrieved documents, but to differentiate the identified confidential documents dependin