US-12619715-B1 - Kernel-level monitoring for software applications

US12619715B1US 12619715 B1US12619715 B1US 12619715B1US-12619715-B1

Abstract

The systems and methods disclosed herein monitor application (e.g., artificial intelligence (AI) model) operations using interactions between the application and a kernel. The systems and methods disclosed herein intercept, using a kernel interface, one or more function invocations transmitted from the application (e.g., an AI model without model modification). Event record(s) are generated for one or more functions to define process identifiers, resource interaction types, timestamps, and/or resource identifiers. Observed pattern(s) for the application are identified by comparing current event record(s) with previous record(s), and the identified observed pattern(s) are evaluated against reference pattern(s) to generate score(s). Data packet(s) that indicate observed pattern(s), corresponding score(s), and/or cryptographic digital fingerprint(s) of the one or more functions are generated. The data packet(s) are transmitted to distributed ledgers for immutable storage.

Inventors

Ganesh Prasad Bhat
James Randolph Myers

Assignees

CITIBANK, N.A.

Dates

Publication Date: 20260505
Application Date: 20251031

Claims (20)

1 . A non-transitory computer-readable storage medium comprising instructions stored thereon for monitoring artificial intelligence (AI) application operations using kernel commands, wherein the instructions when executed by at least one data processor of a system, cause the system to: intercept, using a kernel interface, a function invocation transmitted from an AI application, wherein the kernel interface is configured to manage communications with a kernel of an operating system, and wherein the function invocation represents a request to execute a function set using the kernel; generate an event record for each function of the function set that defines two or more of: a process identifier assigned by a scheduler of the operating system, a type of resource interaction, a timestamp, and an identifier of a resource identified by the function invocation; evaluate the event record for each function of the function set by: identifying an observed pattern set associated with the event record by comparing the event record with one or more previous event records generated from one or more previous functions requested by the AI application and obtained by the kernel interface, and embedding the observed pattern set into a vector representation of the observed pattern set by mapping each observed pattern into a latent space and determining a score set for the event record using a distance between a vector representation of a reference pattern set and the vector representation of the observed pattern set in the latent space; and store, in a kernel-accessible data structure, a data packet for each function of the function set that indicates the score set for the event record, and a digital fingerprint generated by applying a hash function to a representation of the function invocation.
2 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to: in response to a determination that the observed pattern set satisfies a predefined constraint, cause transmission of the function invocation to the kernel interface.
3 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to: in response to a determination that the observed pattern set fails to satisfy a predefined constraint, prevent transmission of the function invocation to the kernel interface.
4 . The non-transitory computer-readable storage medium of claim 1 , wherein one or more of the functions within the function set are configured to: request execution of a file read transaction, request execution of a file write transaction, transmit a network packet, receive the network packet, allocate at least a portion of computer memory, release at least a portion of the computer memory, or initiate a new process thread.
5 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to: cause transmission of each data packet to a distributed ledger, wherein the distributed ledger is configured to store the data packet across multiple nodes, wherein the distributed ledger is a blockchain, and wherein the blockchain is configured to record the data packet as a smart contract across the multiple nodes.
6 . The non-transitory computer-readable storage medium of claim 1 , cause transmission of each data packet to a distributed ledger, wherein the distributed ledger is configured to store the data packet across multiple nodes, wherein the distributed ledger is a federated ledger, and wherein the federated ledger is configured to determine consensus among a predetermined set of authorized nodes prior to storing the data packet.
7 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to: determine that the observed pattern set indicates one or more of: data exfiltration, privilege escalation, lateral movement, or resource consumption that fails to satisfy a particular constraint, and generate an alert on a computing device managed by the operating system in response to the determination.
8 . The non-transitory computer-readable storage medium of claim 1 , wherein the instructions further cause the system to: use an extended Berkeley Packet Filter (eBPF) program to obtain the function invocation, wherein the eBPF program is attached to a network socket, and wherein the function invocation is transmitted over the network socket.
9 . A computer-implemented method for monitoring artificial intelligence (AI) application operations using interactions between the AI application and a kernel, the computer-implemented method comprising: obtaining, using a kernel interface, a function invocation transmitted from an AI application, wherein the kernel interface is configured to manage communications with the kernel of an operating system, and wherein the function invocation represents a request to execute a function set using the kernel; generating an event record for each function of the function set that defines two or more of: a process identifier assigned by a scheduler of the operating system, a type of resource interaction, a timestamp, and an identifier of a resource identified by the function invocation; evaluating the event record for each function of the function set by: identifying an observed pattern set associated with the event record based on comparing the event record with one or more previous event records generated from one or more previous functions requested by the AI application and obtained by the kernel interface, and embedding the observed pattern set into a vector representation of the observed pattern set by mapping each observed pattern into a latent space and determining a score set for the event record using a distance between a vector representation of a reference pattern set and the vector representation of the observed pattern set in the latent space; and storing, in a kernel-accessible data structure, a data packet for each function of the function set that indicates the score set for the event record, and a digital fingerprint generated by applying a hash function to a representation of the function invocation.
10 . The computer-implemented method of claim 9 , wherein obtaining the function invocation comprises: using an extended Berkeley Packet Filter (eBPF) program to intercept the function invocation, wherein the eBPF program is attached to a network socket, and wherein the function invocation is transmitted over the network socket.
11 . The computer-implemented method of claim 9 , further comprising: evaluating the observed pattern set against a reference pattern set; and updating the reference pattern set based on one or more changes determined in the observed pattern set over a time period.
12 . The computer-implemented method of claim 9 , further comprising: determining a score set for the event record by evaluating the observed pattern set against a reference pattern set, wherein the data packet indicates the score set for the event record.
13 . The computer-implemented method of claim 12 , wherein evaluating the event record further comprises: determining a particular score for the observed pattern set across one or more categories, and applying a weight to each particular score for each of the one or more categories to generate the score set.
14 . The computer-implemented method of claim 12 , wherein determining the score set comprises: determining a match score based on a degree of similarity between the observed pattern set and the reference pattern set, determining a risk score based on one or more differences between the observed pattern set and the reference pattern set, determining a predefined trust score for the AI application, and aggregating the match score, the risk score, and the predefined trust score using a predetermined weight set to generate the score set.
15 . The computer-implemented method of claim 9 , further comprising: causing transmission of each data packet to a database, wherein the database is configured to store the data packet.
16 . A system comprising: at least one hardware processor; and at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to: obtain, using a kernel interface, a function invocation transmitted from an AI application, wherein the kernel interface is configured to manage communications with a kernel of an operating system, and wherein the function invocation represents a request to execute a function set using the kernel; determine an event record for each function of the function set that defines two or more of: a process identifier assigned by a scheduler of the operating system, a type of resource interaction, a timestamp, and an identifier of a resource identified by the function invocation; evaluate the event record for each function of the function set using a rule set that is created based on identifying an observed pattern set associated with the event record by comparing the event record with one or more previous event records generated from one or more previous functions requested by the AI application and obtained by the kernel interface, wherein the rule set is stored in a first kernel-accessible data structure; embedding the observed pattern set into a vector representation of the observed pattern set by mapping each observed pattern into a latent space and determining a score set for the event record using a distance between a vector representation of a reference pattern set and the vector representation of the observed pattern set in the latent space; and store, in a second kernel-accessible data structure, a data packet for each function of the function set that indicates the score set for the event record, and a digital fingerprint generated by applying a hash function to a representation of the function invocation.
17 . The system of claim 16 , wherein the system is further caused to: cause transmission of each data packet to a database.
18 . The system of claim 16 , wherein the system is further caused to: apply the rule set is during the evaluation of the event record by comparing the event record against one or more rules within the stored rule set.
19 . The system of claim 16 , wherein the system is further caused to: obtain, from each of a plurality of applications executing on the operating system, a respective function invocation that represents a respective request to execute a respective function using the kernel; access a unique agent identifier corresponding to each of the plurality of applications, wherein each unique agent identifier is mapped to a particular rule set; determine a particular event record for each function; and evaluate each particular event record by comparing the particular event record against the particular rule set mapped to a respective unique agent identifier.
20 . The system of claim 16 , wherein the application is configured to be executed within a sandbox environment that is structured to restrict an access of the application to one or more system resources.

Description

BACKGROUND A kernel is a computer program within an operating system that operates with the highest level of system privileges and provides services to other software running on a computer system. The kernel executes in a protected memory region referred to as kernel space where the kernel has unrestricted access to system resources such as physical memory, processor registers, hardware devices, and so forth. Kernel code can directly manipulate hardware components (e.g., network interfaces, storage controllers, memory management units). The kernel operates as the intermediary between a software application and computer hardware. System calls enable software applications to request services without direct hardware manipulation. A user space represents the restricted execution environment where application programs and user processes operate with limited privileges and controlled access to system resources. User space processes execute in virtual memory spaces that are isolated from each other and from kernel memory to prevent unauthorized access to system resources. Rather than directly accessing hardware devices, modifying system configurations, or manipulating kernel data structures, applications running in user space instead request these services from the kernel via system call interfaces that provide controlled access to system functionality. However, malicious applications can exploit this system call interface to access kernel space resources, by, for example, bypassing user space security controls or manipulating system resources at the kernel level to conceal malicious activities and evade detection by user space monitoring systems. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 shows a schematic illustrating an example environment of an architecture of a kernel-based management platform used to monitor AI agents, in accordance with some implementations of the present technology. FIG. 2 shows a schematic illustrating an example environment of a monitoring layer used by a kernel-based management platform to monitor AI agents, in accordance with some implementations of the present technology. FIG. 3 shows a schematic illustrating an example environment of a behavioral analysis engine used by a kernel-based management platform to score AI agents, in accordance with some implementations of the present technology. FIG. 4 is a flow diagram illustrating an example process of recording AI agent attestations using a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 5 shows a schematic illustrating an example environment of a threat detection module implemented by a kernel-based management platform to detect threatening agent activities, in accordance with some implementations of the present technology. FIG. 6 is a flow diagram illustrating an example process of monitoring AI model operations using a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 7 shows a schematic illustrating an example environment of an architecture of a kernel-based management platform used to manage resources, in accordance with some implementations of the present technology. FIG. 8 shows a schematic illustrating an example environment of an eBPF program implemented by a kernel-based management platform to manage resources, in accordance with some implementations of the present technology. FIG. 9 is a flow diagram illustrating an example process of managing resources using a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 10 shows a schematic illustrating an example environment of adjusting resource costs using a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 11 shows a schematic illustrating an example environment of a hash chain structure implemented by a kernel-based management platform to verify resource usage, in accordance with some implementations of the present technology. FIG. 12 illustrates a block diagram showing an example of a component interaction schema for agent data provenance and lineage tracking in a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 13 is a block diagram illustrating an example architecture for agent data provenance and lineage tracking in a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 14 is a flow diagram illustrating an example process of agent data provenance and lineage tracking using a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 15A is a block diagram showing an example of a system architecture for an agent anomaly detection and automatic quarantine engine used by a kernel-based management platform, in accordance with some implementations of the present technology. FIG. 15B is a block diagram showin