US-12619639-B2 - Systems and methods for implementing permission bypass for large language models

US12619639B2US 12619639 B2US12619639 B2US 12619639B2US-12619639-B2

Abstract

Systems and methods are provided for implementing permission bypass for LLM applications. One system includes an electronic processor that may be configured to receive, from a user device of a user, a user query pertaining to a topic. The electronic processor may also be configured to determine, responsive to a semantic search of a vector database, a plurality of electronic files related to the topic of the user query. The electronic processor may also be configured to determine, based on a permission level of the user, a first portion of the plurality of electronic files, where the first portion of the plurality of electronic files are accessible to the user under the permission level. The electronic processor may also be configured to generate, using a LLM, a response to the user query based on the first portion of the plurality of electronic files.

Inventors

Kyle Robert Kierzyk

Assignees

Boost SubscriberCo L.L.C.

Dates

Publication Date: 20260505
Application Date: 20231208

Claims (17)

1 . A system, the system comprising: one or more electronic processors configured to: receive, from a user device of a user, a user query pertaining to a topic; determine, responsive to a semantic search of a vector database storing a plurality of vector embeddings based on a user query embedding for the user query, a plurality of electronic files related to the topic of the user query; determine, based on a permission level of the user, an accessibility of a first portion of the plurality of electronic files, wherein the first portion of the plurality of electronic files are accessible to the user under the permission level; determine, for the permission level, a first set of vector embeddings of the plurality of vector embeddings, the first set of vector embeddings representing the first portion of the plurality of electronic files being accessible under the permission level; determine a second set of vector embeddings of the plurality of vector embeddings, the second set of vector embeddings representing the plurality of electronic files, wherein the second set of vector embeddings includes the first set of vector embeddings and a third set of vector embeddings representing a second portion of the plurality of electronic files being inaccessible to the user under the permission level; determine a first similarity metric between the first set of vector embeddings and the user query embedding; determine a second similarity metric between the second set of vector embeddings and the user query embedding; determine a difference between the first similarity metric and the second similarity metric; and generate, using a large language model (“LLM”), a response to the user query based on the difference and the first portion of the plurality of electronic files.
2 . The system of claim 1 , wherein the one or more electronic processors are configured to: generate, using an embedding model, the user query embedding for the user query; and execute, based on the user query embedding, the semantic search of the vector database.
3 . The system of claim 1 , wherein the one or more electronic processors are configured to determine the first portion of the plurality of electronic files by matching the user query embedding to a set of vector embeddings of the plurality of vector embeddings, wherein the set of vector embeddings represent the first portion of the plurality of electronic files.
4 . The system of claim 1 , wherein the one or more electronic processors are configured to: determine that the first difference is within a first threshold; and execute a first LLM query using the first portion of the plurality of electronic files, wherein the response to the user query is generated based on the first LLM query executed using the first portion of the plurality of electronic files.
5 . The system of claim 1 , wherein the one or more electronic processors are configured to: determine that the difference exceeds a first threshold; execute a second LLM query using the first portion of the plurality of electronic files; execute a third LLM query using the plurality of electronic files; generate a second LLM query response embedding representing a first output of the second LLM query response; generate a third LLM query response embedding representing a second output of the third LLM query response; and determine a third similarity metric between the second LLM query response embedding and the third LLM query response embedding, wherein the response to the user query is generated based on the third similarity metric.
6 . The system of claim 5 , wherein the one or more electronic processors are configured to: determine that the third similarity metric satisfies a second threshold; and execute, using the LLM, a fourth LLM query using the first portion of the plurality of electronic files; wherein the response to the user query is generated based on execution of the fourth LLM query.
7 . The system of claim 6 , wherein the one or more electronic processors are configured to: generate a notification to indicate that the permission level impacted the response.
8 . The system of claim 6 , wherein the one or more electronic processors are configured to: generate a set of instructions for changing the permission level to a different permission level.
9 . The system of claim 1 , wherein the one or more electronic processors are configured to: train, with training data, the LLM model using machine learning, wherein the LLM model is an artificial neural network.
10 . The system of claim 1 , wherein the LLM model is trained using at least one of self-supervised learning or semi-supervised learning.
11 . A method, the method comprising: receiving, with one or more electronic processors, a user query from a user device of a user, the user query being related to a topic; executing, with the one or more electronic processors, a search of a vector database based on the user query; determining, with the one or more electronic processors, based on the search, electronic content related to the topic of the user query; determining, with the one or more electronic processors, based on a permission level, an accessibility of the electronic content to the user, wherein a first portion of the electronic content is accessible to the user under the permission level and a second portion of the electronic content is inaccessible to the user under the permission level; executing, with the one or more electronic processors, using a large language model “(LLM”), a first LLM query using the first portion of the electronic content; executing, with the one or more electronic processors, using the LLM, a second LLM query using the first portion and the second portion of the electronic content; and generating, with the one or more electronic processors, a first response to the user query based on a similarity of a first output of the first LLM query and a second output of the second LLM query.
12 . The method of claim 11 , further comprising: determining that the first portion of the electronic content is accessible to the user under the permission level; and determining that the second portion of the electronic content is inaccessible to the user under the permission level.
13 . The method of claim 11 , further comprising: determining that the first portion of the electronic content is accessible to the user under the permission level; determining that the second portion of the electronic content is inaccessible to the user under the permission level; generating, using an embedding model, a first LLM query embedding for a first output of the first LLM query; generating, using the embedding model, a second LLM query embedding for a second output of the second LLM query; determining a similarity metric between the first LLM query embedding and the second LLM query embedding; and generating, based on the similarity metric, a fourth response to the user query.
14 . The method of claim 11 , further comprising: generating, using an embedding model, a user query embedding for the user query; and identifying a plurality of vector embeddings from the vector database, the plurality of vector embeddings being within a similarity threshold of the user query embedding, wherein the plurality of vector embeddings represent the electronic content.
15 . A non-transitory, computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions, the set of functions comprising: receiving, from a user device of a user, a user query; generating, using an embedding model, a user query embedding for the user query; executing a semantic search of a vector database to identify, based on the user query embedding, a plurality of vector embeddings from the vector database; determining, based on the plurality of vector embeddings, a plurality of electronic files related to the user query; determining, based on a permission level, an accessibility of the plurality of electronic files for the user; determining, for the permission level, a first set of vector embeddings of the plurality of vector embeddings, the first set of vector embeddings representing a first portion of the plurality of electronic files being accessible under the permission level; determining a second set of vector embeddings of the plurality of vector embeddings, the second set of vector embeddings representing the plurality of electronic files, wherein the second set of vector embeddings includes the first set of vector embeddings and a third set of vector embeddings representing a second portion of the plurality of electronic files being inaccessible to the user under the permission level; determine a difference between a first similarity metric and a second similarity metric, wherein the first similarity metric is between the first set of vector embeddings and the user query embedding and the second similarity metric is between the second set of vector embeddings and the user query embedding; and generate, using a large language model (“LLM”), a response to the user query based on the difference.
16 . The computer-readable medium of claim 15 , wherein the set of functions further comprises: determining an impact of the permission level on responding to the user query, wherein generating the response to the user query includes generating the response to the user query based on the impact.
17 . The computer-readable medium of claim 16 , wherein generating the response to the user query based on the impact includes at least one of: generating the response to the user query based on the first portion of the electronic files; or generating a notification indicating the impact of the permission level on the response.

Description

BACKGROUND This disclosure relates to large language model (“LLM”) applications. LLMs have the ability understand and process text. LLMs generally perform a variety of natural language processing (“NLP”) related tasks to produce content based on input prompts in human language. The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter. SUMMARY This disclosure is in the field of LLM applications, and more particularly, in the field of implementing permission bypass in LLM applications. LLMs have an ability to understand and process text. In some instances, a user can have an LLM help with a problem that contains proprietary information, such as through use of context learning. In some configurations, a vector database may be implemented. A vector database may store processed embeddings of the text (also referred to as vector embeddings). The vector database may store information in a manner such that related data is closer together. When the vector database is queried, the vector database may respond by identifying data that is most similar or useful to the query and providing that data to the LLM. This allows for automated retrieval of context or information for the LLM. Some systems can be implemented such that users can only access information said users have permission to access. Such restricted access can protect sensitive information from being accessed by a user via the LLM. However, by restricting the accessibility of information, the system may fail to identify an insight. Accordingly, the technology disclosed herein may provide a solution to such technical problems. One configuration may provide a system. The system may include one or more electronic processors. The one or more electronic processors may be configured to receive, from a user device of a user, a user query pertaining to a topic. The one or more electronic processors may be configured to determine, responsive to a semantic search of a vector database, a plurality of electronic files related to the topic of the user query. The one or more electronic processors may be configured to determine, based on a permission level of the user, an accessibility of a first portion of the plurality of electronic files, where the first portion of the plurality of electronic files are accessible to the user under the permission level. The one or more electronic processors may be configured to generate, using a large language model (“LLM”), a response to the user query based on the first portion of the plurality of electronic files. Another configuration may provide a method. The method may include receiving, with one or more electronic processors, a user query from a user device of a user, the user query being related to a topic. The method may include executing, with the one or more electronic processors, a search of a vector database based on the user query. The method may include determining, with the one or more electronic processors, based on the search, electronic content related to the topic of the user query. The method may include determining, with the one or more electronic processors, based on a permission level, an accessibility of the electronic content to the user. The method may include, when the electronic content is accessible to the user under the permission level, executing, with the one or more electronic processors, using a large language model “(LLM”), a first LLM query using the electronic content; and generating, with the one or more electronic processors, a first response to the user query based on a first result of the first LLM query. Yet another configuration may provide a non-transitory, computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions. The set of functions may include receiving, from a user device of a user, a user query. The set of functions may include generating, using an embedding model, a user query embedding for the user query. The set of functions may include executing a semantic search of a vector database to identify, based on the user query embedding, a plurality of vector embeddings from the vector database. The set of functions may include determining, based on the plurality of vector embeddings, a plurality of electronic files related to the user query. The set of functions may include determining, based on a permission level, an accessibility of the plurality of electronic files for the user. The set of functions may include generate, using a large language model (“LLM”), a response to the user query based on the accessibility of the plurality of electronic files. This Summary and the Abstract are provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. The Summary and the Abstract are not intended to identify key features or essential features of the claim