US-12626160-B2 - Managing impact of poisoned inferences on deployments of hardware to downstream consumers

US12626160B2US 12626160 B2US12626160 B2US 12626160B2US-12626160-B2

Abstract

Methods and systems for managing impact of inferences provided to inference consumers on decisions made by the inference consumers are disclosed. Poisoned training data may be introduced and used to train an AI model, which may then poison the AI model and lead to poisoned inferences being provided to the inference consumers. Inference consumers may deploy hardware to customers based on the poisoned inferences. To determine whether to modify the deployed hardware, a performance cost associated with the deployed hardware may be obtained. The performance cost may indicate a deviation between operation of the deployed hardware and operation of hardware that may have been deployed if an unpoisoned inference was used. If the performance cost meets a performance cost threshold, at least one additional hardware component may be deployed to the customer.

Inventors

Ofir Ezrielev
Tomer Kushnir
Amihai Savir

Assignees

DELL PRODUCTS L.P.

Dates

Publication Date: 20260512
Application Date: 20230831

Claims (20)

1 . A method of managing an impact of poisoned inferences on downstream consumers, the method comprising: making an identification that a decision by a downstream consumer of the downstream consumers was made based on a poisoned inference of the poisoned inferences, the poisoned inference being generated by a poisoned artificial intelligence (AI) model; estimating a performance cost for the decision based on at least a first operation of a first deployment due to the decision; making a determination regarding whether the performance cost meets a performance cost threshold; and in an instance of the determination in which the performance cost meets the performance cost threshold: performing an action set to manage the first operation of the first deployment by adding at least one additional hardware component to the first deployment.
2 . The method of claim 1 , wherein the decision comprises identifying first hardware components for the first deployment using the poisoned inference and deploying instances of the first hardware components to obtain the first deployment for use by the downstream consumer.
3 . The method of claim 2 , wherein estimating the performance cost for the decision comprises: identifying hardware components for a second deployment using an unpoisoned inference, the unpoisoned inference being a replacement inference for the poisoned inference and generated using an unpoisoned instance of the AI model; obtaining a deviation between the first operation of the first deployment and a second operation of the second deployment; obtaining a confidence score for the deviation, the confidence score indicating a level of uncertainty associated with the first operation and/or the second operation; and treating the deviation and the confidence score as the performance cost.
4 . The method of claim 3 , wherein the second deployment includes second hardware components, the second hardware components being based on a retrospective assessment of needs of the downstream consumer at a time the poisoned inference was made.
5 . The method of claim 3 , wherein making the determination comprises: making a first comparison between the confidence score and a confidence score threshold; and in a first instance of the first comparison in which the confidence score does not meet the confidence score threshold: concluding that the performance cost does not meet the performance cost threshold.
6 . The method of claim 5 , wherein making the determination further comprises: in a second instance of the first comparison in which the confidence score meets the confidence score threshold: making a second comparison between the deviation and a deviation threshold; in a first instance of the second comparison in which the deviation meets the deviation threshold: concluding that the performance cost meets the performance cost threshold; and in a second instance of the second comparison in which the deviation does not meet the deviation threshold: concluding that the performance cost does not meet the performance cost threshold.
7 . The method of claim 3 , wherein performing the action set comprises: identifying a third deployment based, at least in part, on the second deployment; and modifying the first deployment based on the third deployment by adding the at least one additional hardware component to the first deployment.
8 . The method of claim 7 , wherein the third deployment is identified at least in part using a global optimization process that considers the performance cost, a financial cost for adding the at least one additional hardware component to the first deployment, and a reduction in the performance cost due to addition of the at least one additional hardware component to the first deployment.
9 . The method of claim 8 , wherein modifying the first deployment comprises shipping the at least one additional hardware component to the first deployment and installing the at least one additional hardware component with the first hardware components of the first deployment.
10 . A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing an impact of inferences provided to an inference consumer on operation of the inference consumer, the operations comprising: making an identification that a decision by a downstream consumer of the downstream consumers was made based on a poisoned inference of the poisoned inferences, the poisoned inference being generated by a poisoned artificial intelligence (AI) model; estimating a performance cost for the decision based on at least a first operation of a first deployment due to the decision; making a determination regarding whether the performance cost meets a performance cost threshold; and in an instance of the determination in which the performance cost meets the performance cost threshold: performing an action set to manage the first operation of the first deployment by adding at least one additional hardware component to the first deployment.
11 . The non-transitory machine-readable medium of claim 10 , wherein the decision comprises identifying first hardware components for the first deployment using the poisoned inference and deploying instances of the first hardware components to obtain the first deployment for use by the downstream consumer.
12 . The non-transitory machine-readable medium of claim 11 , wherein estimating the performance cost for the decision comprises: identifying hardware components for a second deployment using an unpoisoned inference, the unpoisoned inference being a replacement inference for the poisoned inference and generated using an unpoisoned instance of the AI model; obtaining a deviation between the first operation of the first deployment and a second operation of the second deployment; obtaining a confidence score for the deviation, the confidence score indicating a level of uncertainty associated with the first operation and/or the second operation; and treating the deviation and the confidence score as the performance cost.
13 . The non-transitory machine-readable medium of claim 12 , wherein the second deployment includes second hardware components, the second hardware components being based on a retrospective assessment of needs of the downstream consumer at a time the poisoned inference was made.
14 . The non-transitory machine-readable medium of claim 12 , wherein making the determination comprises: making a first comparison between the confidence score and a confidence score threshold; and in a first instance of the first comparison in which the confidence score does not meet the confidence score threshold: concluding that the performance cost does not meet the performance cost threshold.
15 . The non-transitory machine-readable medium of claim 14 , wherein making the determination further comprises: in a second instance of the first comparison in which the confidence score meets the confidence score threshold: making a second comparison between the deviation and a deviation threshold; in a first instance of the second comparison in which the deviation meets the deviation threshold: concluding that the performance cost meets the performance cost threshold; and in a second instance of the second comparison in which the deviation does not meet the deviation threshold: concluding that the performance cost does not meet the performance cost threshold.
16 . A data processing system, comprising: a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing an impact of inferences provided to an inference consumer on operation of the inference consumer, the operations comprising: making an identification that a decision by a downstream consumer of the downstream consumers was made based on a poisoned inference of the poisoned inferences, the poisoned inference being generated by a poisoned artificial intelligence (AI) model; estimating a performance cost for the decision based on at least a first operation of a first deployment due to the decision; making a determination regarding whether the performance cost meets a performance cost threshold; and in an instance of the determination in which the performance cost meets the performance cost threshold: performing an action set to manage the first operation of the first deployment by adding at least one additional hardware component to the first deployment.
17 . The data processing system of claim 16 , wherein the decision comprises identifying first hardware components for the first deployment using the poisoned inference and deploying instances of the first hardware components to obtain the first deployment for use by the downstream consumer.
18 . The data processing system of claim 17 , wherein estimating the performance cost for the decision comprises: identifying hardware components for a second deployment using an unpoisoned inference, the unpoisoned inference being a replacement inference for the poisoned inference and generated using an unpoisoned instance of the AI model; obtaining a deviation between the first operation of the first deployment and a second operation of the second deployment; obtaining a confidence score for the deviation, the confidence score indicating a level of uncertainty associated with the first operation and/or the second operation; and treating the deviation and the confidence score as the performance cost.
19 . The data processing system of claim 18 , wherein the second deployment includes second hardware components, the second hardware components being based on a retrospective assessment of needs of the downstream consumer at a time the poisoned inference was made.
20 . The data processing system of claim 18 , wherein making the determination comprises: making a first comparison between the confidence score and a confidence score threshold; and in a first instance of the first comparison in which the confidence score does not meet the confidence score threshold: concluding that the performance cost does not meet the performance cost threshold.

Description

FIELD Embodiments disclosed herein relate generally to artificial intelligence (AI) models. More particularly, embodiments disclosed herein relate to systems and methods to manage impact of inferences generated by AI models on decisions made by downstream consumers 7 of the inferences. BACKGROUND Computing devices may provide computer-implemented services. The computer-implemented services may be used by users of the computing devices and/or devices operably connected to the computing devices. The computer-implemented services may be performed with hardware components such as processors, memory modules, storage devices, and communication devices. The operation of these components and the components of other devices may impact the performance of the computer-implemented services. BRIEF DESCRIPTION OF THE DRAWINGS Embodiments disclosed herein are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements. FIG. 1 shows a block diagram illustrating a system in accordance with an embodiment. FIG. 2A shows a data flow diagram illustrating an AI model manager in accordance with an embodiment. FIG. 2B shows a data flow diagram illustrating an AI model manager generating a replacement inference for a poisoned inference in accordance with an embodiment. FIG. 2C shows a data flow diagram illustrating an AI model manager managing impact of a poisoned inference on decisions made by an inference consumer in accordance with an embodiment. FIG. 3A shows a flow diagram illustrating a method of updating an AI model instance in accordance with an embodiment. FIG. 3B shows a flow diagram illustrating a method of managing poisoned training data in accordance with an embodiment. FIG. 3C shows a flow diagram illustrating a method of managing impact of a poisoned inference on decisions made by an inference consumer in accordance with an embodiment. FIG. 4 shows a block diagram illustrating a data processing system in accordance with an embodiment. DETAILED DESCRIPTION Various embodiments will be described with reference to details discussed below, and the accompanying drawings will illustrate the various embodiments. The following description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of various embodiments. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments disclosed herein. Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment. The appearances of the phrases “in one embodiment” and “an embodiment” in various places in the specification do not necessarily all refer to the same embodiment. References to an “operable connection” or “operably connected” means that a particular device is able to communicate with one or more other devices. The devices themselves may be directly connected to one another or may be indirectly connected to one another through any number of intermediary devices, such as in a network topology. In general, embodiments disclosed herein relate to methods and systems for managing impact of inferences generated by AI models on decisions made by inference consumers. Trained AI models may provide computer-implemented services (e.g., inference generation) for downstream consumers of the inferences (e.g., inference consumers). To manage trained AI models, a data processing system may, over time, update AI models through training using training data. However, if poisoned training data is introduced to an AI model, the AI model may become untrustworthy (e.g., the AI model may be tainted by the poisoned training data). Inferences generated using the tainted (e.g., poisoned) AI model may also be untrustworthy or inaccurate. Once it has been discovered that an AI model has been tainted with poisoned training data, the model may require re-training to remove the influence of the poisoned training data, and any or all inferences generated using the tainted AI model may be untrustworthy. Training an AI model may be a computationally expensive process and may require the use of a limited amount of computing resources that may otherwise be used for inference generation (and/or other purposes). In other words, computing resources spent re-training AI models may interrupt inference consumption and/or other types of computer-implemented services that may otherwise be provided using the computing resources dedicated to re-training. To reduce computing resources spent re-training AI models, an AI model snapshot may be obtained periodically throughout the AI model training process. The snapshot may store information regarding the structure of the AI model, which may be used to restore a partially