US-12626098-B2 - Method and system for creating an ensemble of neural network-based classifiers that optimizes a diversity metric

Abstract

One embodiment provides a system which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers. The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers. The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric. The system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.

Inventors

• Alejandro E. Brito
• Shantanu Rane

Assignees

• PALO ALTO RESEARCH CENTER INCORPORATED

Dates

Publication Date

20260512

Application Date

20220915

Claims (20)

1. 1 . A computer-executable method, comprising: defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers; including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers; replacing a final classification layer in one or more neural networks with the three or more affine classifiers; training, based on a training data object or a testing data object, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric; predicting an output for a respective data object based on the ensemble of neural network-based classifiers which optimize the diversity metric; displaying, on a device associated with a user, information related to the predicted output for the respective data object, the information comprising one or more of: a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack; a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack; a size of the ensemble of neural network-based classifiers; an approach associated with the respective data object, including whether the respective data object is associated with diversity; a type of classifier used in a final classifier layer; the predicted outcome based on a respective classifier; and an overall ensemble result; receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classifier layer, a selected classifier, and the regularization term; predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.
2. 2 . The method of claim 1 , wherein one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.
3. 3 . The method of claim 1 , wherein a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.
4. 4 . The method of claim 1 , wherein the three or more affine classifiers comprise a multi-class classification layer.
5. 5 . The method of claim 1 , wherein the three or more affine classifiers comprise a one-versus-all classification layer.
6. 6 . The method of claim 1 , wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.
7. 7 . The method of claim 1 , wherein the three or more affine classifiers comprise an odd number of affine classifiers, and wherein the method further comprises generating a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.
8. 8 . The method of claim 1 , further comprising: predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.
9. 9 . The method of claim 8 , further comprising: wherein predicting the outcome for the testing data object is further based an ensemble decision rule.
10. 10 . The method of claim 1 , wherein defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and wherein training the one or more neural networks is further based on the training data.
11. 11 . A computer system for facilitating data classification, the computer system comprising: a processor; and a storage device storing instructions that when executed by the processor cause the processor to perform a method, the method comprising: defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers; including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers; replacing a final classification layer in one or more neural networks with the three or more affine classifiers; training, based on a training data object or a testing data object, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric; predicting an outcome for a respective data object based on the ensemble of neural network-based classifiers which optimize the diversity metric; displaying, on a device associated with a user, information related to the predicted outcome for the respective data object, the information comprising one or more of: a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack; a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack; a size of the ensemble of neural network-based classifiers; an approach associated with the respective data object, including whether the respective data object is associated with diversity; a type of classifier used in a final classifier layer; the predicted outcome based on a respective classifier; and an overall ensemble result; receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classification layer, a selected classifier, and the regularization term; predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.
12. 12 . The computer system of claim 11 , wherein one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.
13. 13 . The computer system of claim 11 , wherein a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.
14. 14 . The computer system of claim 11 , wherein the three or more affine classifiers comprise at least one of a multi-class classification layer and a one-versus-all classification layer.
15. 15 . The method of claim 11 , wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.
16. 16 . The computer system of claim 11 , wherein the three or more affine classifiers comprise an odd number of affine classifiers, and wherein the method further comprises generating a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.
17. 17 . The computer system of claim 11 , wherein the method further comprises: predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.
18. 18 . The computer system of claim 11 , wherein defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and wherein training the one or more neural networks is further based on the training data.
19. 19 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising: defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers for training data; including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers; replacing a final classification layer in one or more neural networks with the three or more affine classifiers; training, based on the training data, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric; predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric; displaying, on a device associated with a user, information related to the predicted outcome for a respective data object, the information comprising one or more of: a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack; a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack; a size of the ensemble of neural network-based classifiers; an approach associated with the respective data object, including whether the respective data object is associated with diversity; a type of classifier used in a final classifier layer; the predicted outcome based on a respective classifier; and an overall ensemble result; receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classification layer, a selected classifier, and the regularization term; predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.
20. 20 . The non-transitory computer-readable storage medium of claim 19 , wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal, wherein at least one of the following: one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers; and a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers, and wherein the three or more affine classifiers comprise at least one of a multi-class classification layer and a one-versus-all classification layer.

Description

RELATED APPLICATIONS This application is related to: U.S. application Ser. No. 17/158,631, entitled “System and Method for Reasoning About the Diversity and Robustness of an Ensemble of Classifiers,” by inventors Shantanu Rane, Alejandro E. Brito, and Hamed Soroush, filed 26 Jan. 2021 (hereinafter “application. Ser. No. 17/158,631”); U.S. application Ser. No. 17/345,996, entitled “Method and System for Creating an Ensemble of Machine Learning Models to Defend Against Adversarial Examples,” by inventors Alejandro E. Brito, Bashir Sadeghi, and Shantanu Rane, filed 11 Jun. 2021 (hereinafter “application. Ser. No. 17/345,996”); U.S. application Ser. No. 17/400,016, entitled “Method and System for Learning an Ensemble of Neural Network Kernel Classifiers Based on Partitions of the Training Data,” by inventors Alejandro E. Brito, Bashir Sadeghi, and Shantanu Rane, filed 11 Aug. 2021 (hereinafter “application. Ser. No. 17/400,016”); and U.S. application Ser. No. 17/944,939, entitled “Method and System for Optimizing a Pair of Affine Classifiers Based on a Diversity Metric,” by inventors Shantanu Rane, Bashir Sadeghi, and Alejandro E. Brito, filed 14 Sep. 2022 (hereinafter “application. Ser. No. 17/944,939”) the disclosures of which are herein incorporated by reference in their entirety. BACKGROUND Field This disclosure is generally related to machine learning and data classification. More specifically, this disclosure is related to a method and system for creating an ensemble of neural network-based classifiers that optimizes a diversity metric. Related Art In the field of machine learning, adversarial examples can exploit the way that artificial intelligence algorithms work in order to disrupt the behavior of the algorithms. Recently, an increasing number and types of attacks have been devised in order to fool the algorithms, along with increasingly stronger defenses against such attacks. One large class of these attacks is “perturbation-bounded evasion attacks,” which involve adversarial examples constructed by perturbing data samples with the goal of forcing a classifier to misclassify them. Such evasion attacks comprise a predominant class of attacks considered in current machine learning technology. One specific type of evasion attack involves adversarial examples which can be trivially classified by a human but can fool a machine learning classifier. One solution to address these evasion attacks is to use an ensemble or collection of classifiers. However, a principled analysis based on linear models derived from convolutional neural networks (CNNs) remains a challenge. SUMMARY One embodiment provides a system which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers. The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair (i.e., each pair) of mutually orthogonal affine classifiers of the three or more affine classifiers. The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric. In some embodiments, one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers. In some embodiments, a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers. In some embodiments, the three or more affine classifiers comprise a multi-class classification layer. In some embodiments, the three or more affine classifiers comprise a one-versus-all classification layer. In some embodiments, the decision boundaries of the plurality of affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal. In some embodiments, the three or more affine classifiers comprise an odd number of affine classifiers. The system generates a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers. In some embodiments, the system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric. In some embodiments, predicting the outcome for the testing data object is further based an ensemble decision rule. In some embodiments, defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and training the one or more neural networks is further based on the training data. BRIEF DESCRIPTION OF THE FIGURES FIG. 1 presents an exemplary environment which facilitates const