Search

US-20260127265-A1 - Cybersecurity Provenance of ML/AI Models

US20260127265A1US 20260127265 A1US20260127265 A1US 20260127265A1US-20260127265-A1

Abstract

A cybersecurity model assessment service assesses machine learning and/or artificial intelligence models for cybersecurity threats. When an endpoint client device encounters an ML/AI model, the client device may stop processing the ML/AI model and determine its provenance. The provenance identifies a base, foundational, or origin model from which the ML/AI model derives. The provenance, for example, determines whether the ML/AI model originates from, derives from, or is sufficiently similar to a known good/safe model or to a known bad/unsafe model. The cybersecurity model assessment service may then predict a computer behavior of the ML/AI model, based on the provenance. Similarity to a known good/safe model, for example, may be safe to run, while similarity to a known bad/unsafe model is unsafe to run.

Inventors

  • Andrew Southgate
  • Alexandru Dinu
  • Dragos Georgian Corlãtescu
  • Ioana Croitoru

Assignees

  • CROWDSTRIKE, INC.

Dates

Publication Date
20260507
Application Date
20241101

Claims (20)

  1. 1 . A method executed by a computer system that assesses an artificial intelligence (AI) model, comprising: conducting, by the computer system, a sequence of different model provenance tests associated with a digital cybersecurity service that assesses a provenance associated with the AI model; when the AI model satisfies a model provenance test in the sequence of the different model provenance tests, then determining, by the computer system, an operational behavior associated with the provenance; and when the AI model fails to satisfy the model provenance test in the sequence of the different model provenance tests, then conducting, by the computer system, another model provenance test in the sequence of the different model provenance tests that assesses the provenance associated with the AI model.
  2. 2 . The method of claim 1 , further comprising generating a cybersecurity prediction based on the operational behavior associated with the provenance.
  3. 3 . The method of claim 1 , further comprising assigning the operational behavior to the AI model.
  4. 4 . The method of claim 1 , further comprising determining the AI model fails to satisfy the sequence of the different model provenance tests.
  5. 5 . The method of claim 1 , further comprising determining the provenance by comparing file hash values associated with the AI model to historical file hash values associated with known AI models previously assessed.
  6. 6 . The method of claim 5 , further comprising determining the file hash values associated with the AI model match the historical file hash values associated with a known AI model of the known AI models previously assessed.
  7. 7 . The method of claim 6 , wherein in response to the determining that the file hash values associated with the AI model match the historical file hash values associated with the known AI model, then further comprising assigning the operational behavior associated with the known AI model to the AI model.
  8. 8 . A computer system that that assesses an artificial intelligence (AI) model, comprising: at least one central processing unit; and at least one memory device storing instructions that, when executed by the at least one central processing unit, perform operations, the operations comprising: receiving file hash values associated with the AI model reported via a cloud computing environment by a cybersecurity sensory agent installed at a client device; determining a provenance associated with the AI model by conducting a preliminary model provenance test associated with a digital cybersecurity service that compares the file hash values to historical file hash values associated with known AI models; when the file hash values associated with the AI model match the historical file hash values associated with a known AI model of the known AI models, then determining an operational behavior associated with the known AI model; and when the file hash values associated with the AI model fail to match the historical file hash values associated with the known AI model, then determining the provenance associated with the AI model by conducting subsequent model provenance testing.
  9. 9 . The computer system of claim 8 , wherein the operations further comprise determining the AI model is abnormal operation based on the provenance.
  10. 10 . The computer system of claim 8 , wherein the operations further comprise determining the AI model is normal operation based on the provenance.
  11. 11 . The computer system of claim 8 , wherein the operations further comprise sending a cybersecurity prediction generated based on the provenance via the cloud computing environment to the cybersecurity sensory agent installed at the client device.
  12. 12 . The computer system of claim 8 , wherein the operations further comprise blocking or allowing the AI model based on the provenance.
  13. 13 . The computer system of claim 8 , wherein the operations further comprise blocking or allowing the AI model based on the operational behavior associated with the known AI model.
  14. 14 . The computer system of claim 8 , wherein the operations for the conducting of the subsequent model provenance testing further comprise conducting a sequence of different model provenance tests.
  15. 15 . The computer system of claim 8 , wherein the operations further comprise randomly conducting the subsequent model provenance testing.
  16. 16 . A memory device storing instructions that, when executed by at least one central processing unit, perform operations, comprising: receiving file hash values associated with an AI model reported via a cloud computing environment by a cybersecurity sensory agent installed at a client device; determining a provenance associated with the AI model by conducting a preliminary model provenance test associated with a digital cybersecurity service that compares the file hash values to entries in a database of models that map historical file hash values to known AI models previously assessed by the digital cybersecurity service; if the file hash values associated with the AI model match the historical file hash values mapped by the database of models to a known AI model of the known AI models, then identifying an operational behavior mapped by the database of models to the known AI model; sending a cybersecurity prediction via the cloud computing environment to the client device that instructs the cybersecurity sensory agent to allow or block the AI model based on the operational behavior mapped by the database of models to the known AI model; and if the file hash values associated with the AI model fail to match the historical file hash values mapped by the database of models to the known AI models, then determining the provenance associated with the AI model by conducting subsequent model provenance testing.
  17. 17 . The memory device of claim 16 , wherein the operations further comprise determining the operational behavior is normal or abnormal based on the operational behavior mapped by the database of models to the known AI model.
  18. 18 . The memory device of claim 16 , wherein the operations for the conducting of the subsequent model provenance testing further comprise conducting a sequence of different model provenance tests.
  19. 19 . The memory device of claim 16 , wherein the operations further comprise randomly conducting the subsequent model provenance testing.
  20. 20 . The memory device of claim 16 , wherein the operations for the conducting of the subsequent model provenance testing further comprise: sending a model similarity instruction to the client device that instructs the cybersecurity sensory agent to execute a local similarity analysis; receiving similarity values representing the AI model generated by the cybersecurity sensory agent; and determining the provenance associated with the AI model by comparing the similarity values to the entries in the database of models that map historical similarity values to the known AI models previously assessed by the digital cybersecurity service.

Description

BACKGROUND The subject matter described herein generally relates to computers and to computer security and, more particularly, the subject matter relates to artificial neural networks. Cybersecurity threats are always increasing. It seems every day there is another cybersecurity attack that steals account passwords, business data, and personal information. Emails, websites, and text messages often contain malicious links, viruses, and attachments. Now, even machine learning and artificial intelligence are being targeted by cyberattackers. SUMMARY A cybersecurity model assessment service assesses machine learning and/or artificial intelligence models for cybersecurity threats. When an endpoint client device encounters an ML/AI model, the client device may stop processing the ML/AI model and determine its provenance. The provenance identifies a base, foundational, or origin model from which the ML/AI model derives. The provenance, for example, determines whether the ML/AI model originates from, derives from, or is sufficiently similar to a known good/safe model or to a known bad/unsafe model. The cybersecurity model assessment service may then predict a computer behavior of the client device executing the ML/AI model, based on the provenance. For example, if the ML/AI model is similar to a known good/safe model, then the client device may be predicted to safely execute the ML/AI model. If, however, the ML/AI model is similar to a known bad/unsafe model, then the ML/AI model may be predicted as unsafe to execute. As machine learning and artificial intelligence grow in use, the cybersecurity model assessment service protects client devices from newly-emerging cybersecurity threats related to unsafe model usage. BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS The features, aspects, and advantages of predictive cybersecurity provenance of ML/AI models are understood when the following Detailed Description is read with reference to the accompanying drawings, wherein: FIGS. 1-3 illustrate some examples of machine learning (or ML) and artificial intelligence (or AI) model provenance; FIG. 4 illustrates examples of intellectual property protection; FIGS. 5-7 illustrate more examples of a cybersecurity service; FIG. 8 illustrates still more examples of the cybersecurity service and the cybersecurity model assessment service; FIG. 9 illustrates more examples of the cybersecurity model assessment service; FIGS. 10-11 and 12A-12C illustrate examples of the cybersecurity model assessment service and preliminary model provenance testing; FIGS. 13-16 illustrate examples of the cybersecurity model assessment service and subsequent model provenance testing; FIG. 17 illustrates examples of sequential model provenance testing; FIG. 18 illustrates examples of numerically counting model provenance testing; FIG. 19 illustrates examples of random model provenance testing to avoid obfuscation; FIGS. 20-21 illustrate more examples of the cybersecurity model assessment service; FIG. 22 illustrates examples of host monitoring; FIG. 23 illustrates examples of client assessment; FIGS. 24-26 illustrate examples of methods or operations that assess ML/AI models; and FIG. 27 illustrates a more detailed example of an operating environment. DETAILED DESCRIPTION Some examples relate to detecting and predicting abnormal and malicious machine learning (or ML) and artificial intelligence (or AI) models. As we know, nearly every day we read of another network hack, computer virus, or other cybersecurity threat. As machine learning and artificial intelligence grow in usage, cyberattackers are expected to target ML/AI models. Infected ML/AI models will cause new and unexpected cyberthreats. A cybersecurity model assessment service, however, protects computers from new and unexpected ML/AI threats. The cybersecurity model assessment service assesses many different ML/AI models encountered in personal computing and in cloud services. The cybersecurity model assessment service assesses each ML/AI model for its provenance to known good/safe models or to known bad/unsafe models. The provenance, for example, determines whether the ML/AI model originates from, derives from, or is sufficiently similar to one of the known models. The cybersecurity model assessment service may then predict a computer behavior (such as normal or abnormal/malicious), based on the provenance. The cybersecurity model assessment service may thus maintain a library or catalog of the many different ML/AI models likely to be encountered in the field. When a computer encounters an ML/AI model, the computer may stop and check whether the ML/AI model is safe to run. The computer, for example, merely requests the cybersecurity model assessment service and receives a response. If the cybersecurity model assessment service predicts that the ML/AI model is safe to run, then the computer resumes executing the ML/AI model. If, however, the cybersecurity model assessment service predict