US-12619914-B2 - Apparatus and methods for artificial intelligence model management

US12619914B2US 12619914 B2US12619914 B2US 12619914B2US-12619914-B2

Abstract

An artificial intelligence (AI) model management system is disclosed. The system may receive data, such as image data, as part of a task to be completed. The system may apply an AI model, such as a machine learning algorithm, to the data. The system may also generate performance data based on performance of the AI model, and determine whether to route AI model results for verification. During verification, one or more inputs are received indicating whether the AI model results are correct and, if not correct, a correct result. The system may determine which inputs indicate a correct result if, for example, the inputs do not indicate the same correct result. The AI model may be trained in real-time based on the verified or corrected results. In some examples, the system causes one or more actions to be taken upon completion of the task.

Inventors

Daniel William BRUCE
Santiago Luis Valdarrama del Pino

Assignees

LEVATAS, INC.

Dates

Publication Date: 20260505
Application Date: 20211022

Claims (20)

1 . At least one computing device comprising at least one processor, the at least one computing device configured to: receive input data from at least one device; determine at least one classification based on applying an artificial intelligence model to the received input data; determine a performance of the artificial intelligence model based on previous classifications made by the artificial intelligence model; and determine whether to provide the classification for verification based on the performance of the artificial intelligence model, wherein: if the determination is to provide the classification for verification, the classification is provided for user verification; and if the determination is to not provide the classification for verification, at least one action is taken based on the classification.
2 . The at least one computing device of claim 1 , wherein if the determination is to provide the classification for verification, the at least one computing device is further configured to: provide for display the classification to at least one user; receive input data from the at least one user identifying whether the classification is correct; and determine whether the classification is correct based on the received input data.
3 . The at least one computing device of claim 2 , wherein the input data identifies that the classification is not correct, and further identifies the correct classification.
4 . The at least one computing device of claim 1 wherein if the determination is to not provide the classification for verification, the at least one computing device is further configured to: determine whether to provide the classification for verification based on at least one of: one or more rules; and an estimated cost of using the classification and an estimated cost associated with providing the classification for verification.
5 . The at least one computing device of claim 1 wherein the at least one action comprises transmitting a communication to a second computing device.
6 . The at least one computing device of claim 5 , wherein the communication identifies at least one of a status or condition of a system.
7 . The at least one computing device of claim 1 , wherein applying the artificial intelligence model to the received input data comprises determining one of a plurality of artificial intelligence models based on an operating mode, and applying the determined artificial intelligence model to the received input data.
8 . The at least one computing device of claim 1 , wherein determining the performance of the artificial intelligence model comprises receiving performance data indicating system performance of a system.
9 . The at least one computing device of claim 1 , wherein the at least one computing device is configured to: determine a time threshold based on the received input data; and determine whether to provide the classification for verification based on the time threshold.
10 . The at least one computing device of claim 9 , wherein the at least one computing device is configured to: provide the classification for user verification if a verification response can be received within the time threshold; and take the at least one action if the verification response cannot be received within the time threshold.
11 . A method by at least one computing device the method comprising: receiving input data from at least one device; determining at least one classification based on applying an artificial intelligence model to the received input data; determining a performance of the artificial intelligence model based on previous classifications made by the artificial intelligence model; and determining whether to provide the classification for verification based on the performance of the artificial intelligence model, wherein: if the determination is to provide the classification for verification, the classification is provided for user verification; and if the determination is to not provide the classification for verification, at least one action is taken based on the classification.
12 . The method of claim 11 , wherein if the determination is to provide the classification for verification, the method further comprises: providing for display the classification to at least one user; receiving input data from the at least one user identifying whether the classification is correct; and determining whether the classification is correct based on the received input data.
13 . The method of claim 11 , wherein if the determination is to not provide the classification for verification, the method further comprises: determining whether to provide the classification for verification based on at least one of: one or more rules; and an estimated cost of using the classification and an estimated cost associated with providing the classification for verification.
14 . The method of claim 11 , wherein applying the artificial intelligence model to the received input data comprises determining one of a plurality of artificial intelligence models based on an operating mode, and applying the determined artificial intelligence model to the received input data.
15 . The method of claim 11 , further comprising: determining a time threshold based on the received input data; determining whether to provide the classification for verification based on the time threshold; providing the classification for user verification if a verification response can be received within the time threshold; and taking the at least one action if the verification response cannot be received within the time threshold.
16 . A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising: receiving input data from at least one device; determining at least one classification based on applying an artificial intelligence model to the received input data; determining a performance of the artificial intelligence model based on previous classifications made by the artificial intelligence model; and determining whether to provide the classification for verification based on the performance of the artificial intelligence model, wherein: if the determination is to provide the classification for verification, the classification is provided for user verification; and if the determination is to not provide the classification for verification, at least one action is taken based on the classification.
17 . The non-transitory computer readable medium of claim 16 , wherein if the determination is to provide the classification for verification, the operations further comprise: providing for display the classification to at least one user; receiving input data from the at least one user identifying whether the classification is correct; and determining whether the classification is correct based on the received input data.
18 . The non-transitory computer readable medium of claim 16 , wherein if the determination is to not provide the classification for verification, the operations further comprise: determining whether to provide the classification for verification based on at least one of: one or more rules; and an estimated cost of using the classification and an estimated cost associated with providing the classification for verification.
19 . The non-transitory computer readable medium of claim 16 , wherein applying the artificial intelligence model to the received input data comprises determining one of a plurality of artificial intelligence models based on an operating mode, and applying the determined artificial intelligence model to the received input data.
20 . The non-transitory computer readable medium of claim 16 , wherein the operations further comprise: determining a time threshold based on the received input data; determining whether to provide the classification for verification based on the time threshold; providing the classification for user verification if a verification response can be received within the time threshold; and taking the at least one action if the verification response cannot be received within the time threshold.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a § 371 national stage entry of International Application No. PCT/US2021/056169, filed on Oct. 22, 2021, which claims priority to U.S. Provisional Application Ser. No. 63/104,807, filed on Oct. 23, 2020 and entitled “APPARATUS AND METHODS FOR ARTIFICIAL INTELLIGENCE MODEL MANAGEMENT,” the entire contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD This disclosure relates generally to artificial intelligence (AI) model based systems and, more specifically, to the implementation and management of AI models in AI model based systems. BACKGROUND Systems, especially large systems, may include various components. For example, manufacturing systems, electrical systems (e.g., energy producing systems), defense systems, surveillance systems, and transportation systems (e.g., vehicles), among many others, may include various components that work together to produce a result (e.g., an output, a product, etc.). Often times, however, one or more of the components of a system may fail, causing the system to stall or even shut down. At least some systems require the supervision of one or more technicians to assure system components are running without fault. Additionally, some systems may employ component fault detection mechanisms that detect when a component needs service or repair. However, in each case, the systems may need to be shut down for repair once a faulty component is identified, resulting in unexpected shutdown of the system. For example, a system may need to be shut down to fix a faulty component during a temporal period that, otherwise, the system would have spent running to produce results. As such, there are opportunities to improve fault detection in systems. SUMMARY In some examples, at least one computing device is configured to receive input data from at least one device, and determine at least one classification based on applying an artificial intelligence model to the received input data. The, at least one computing device is also configured to determine a performance of the artificial intelligence model based on previous classifications made by the artificial intelligence mode. Further, the at least one computing device is configured to determine whether to provide the classification for verification based on the performance of the artificial intelligence model. If the determination is to provide the classification for verification, the at least one computing device is configured to provide the classification for user verification. If, however, the determination is to not provide the classification for verification, the at least one computing device is configured to take at least one action based on the classification. In some examples, a method by one or more computing devices includes receiving input data from at least one device, and determining at least one classification based on applying an artificial intelligence model to the received input data. The method also includes determining a performance of the artificial intelligence model based on previous classifications made by the artificial intelligence mode. Further, the method includes determining whether to provide the classification for verification based on the performance of the artificial intelligence model. If the determination is to provide the classification for verification, the method includes providing the classification for user verification. If, however, the determination is to not provide the classification for verification, the method includes taking at least one action based on the classification. In some examples, a non-transitory computer readable medium stores instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations including receiving input data from at least one device, and determining at least one classification based on applying an artificial intelligence model to the received input data. The operations also include determining a performance of the artificial intelligence model based on previous classifications made by the artificial intelligence mode. Further, the operations include determining whether to provide the classification for verification based on the performance of the artificial intelligence model. If the determination is to provide the classification for verification, the operations includes providing the classification for user verification. If, however, the determination is to not provide the classification for verification, the operations include taking at least one action based on the classification. In some examples, at least one computing device is configured to receive a task from at least one device, and determine at least one classification based on applying an artificial intelligence model to input data identified by the task. The at least one computing device is also configured to determine a time threshold for the task. Further, the