US-12626125-B2 - Synthesizing a singular ensemble machine learning model from an ensemble of models

Abstract

The present disclosure relates to systems and methods for generating and using a singular ensemble model.

Inventors

• Vincent Pham
• Mark Watson
• Jeremy Goodsitt
• Reza Farivar
• Austin Walters
• Kenneth Taylor
• Fardin Abdi Taghi Abad
• Anh Truong

Assignees

• CAPITAL ONE SERVICES, LLC

Dates

Publication Date

20260512

Application Date

20211001

Claims (20)

1. 1 . A system for improving efficiency of machine learning, comprising: at least one processor; and at least one storage medium storing instructions that, when executed, configure the at least one processor to perform operations comprising: obtaining a plurality of machine learning models; obtaining a training data set; applying the plurality of machine learning models to at least a portion of the training data set to obtain outputs associated with the plurality of machine learning models; generating, based on whether the plurality of machine learning models are of a same model type, a singular machine learning model by combining one or more models, at an output layer, to generate the singular machine learning model, wherein the singular machine learning model has the same model type or a neural network type based on whether the plurality of machine learning models are of the same model type, and wherein the singular machine learning model overfits the plurality of machine learning models; training the singular machine learning model using at least another portion of the training data set, wherein the trained singular machine learning model is executable locally on a client device without needing to transmit inputs to a remote server to obtain results; and outputting the trained singular machine learning model transmitting the trained singular machine learning model to the client device, the client device being configured to: store a data structure defining parameters of the trained singular machine learning model; retrieve input data for the trained singular machine learning model; locally apply the trained singular machine learning model to the input data to generate results without resort to the at least one processor or the at least one storage medium; and display the results.
2. 2 . The system of claim 1 , wherein the plurality of machine learning models comprise at least one neural network or at least one linear regression.
3. 3 . The system of claim 1 , wherein: the singular machine learning model comprises a plurality of layers; and a number of the plurality of layers is at least as great as a number of layers in a model, of the plurality of machine learning models, having a largest number of layers.
4. 4 . The system of claim 1 , wherein each layer of the singular machine learning model comprises a plurality of nodes, a number of the plurality of nodes being at least as great as a number of nodes in corresponding layers in one of the plurality of machine learning models having a largest number of nodes.
5. 5 . The system of claim 1 , wherein the operations further comprise: applying one or more weights to the outputs.
6. 6 . The system of claim 5 , wherein the one or more weights are equal to each other.
7. 7 . The system of claim 5 , wherein the one or more weights comprise inputs from a user.
8. 8 . The system of claim 1 , wherein outputting the trained singular machine learning model comprises of storing the trained singular machine learning model in the at least one storage medium.
9. 9 . The system of claim 1 , further comprising a model clusterer configured to obtain the plurality of machine learning models from a model generator and automatically generate clusters of the plurality of machine learning models.
10. 10 . The system of claim 1 , wherein the operations further comprise: applying a transformation to at least part of the singular machine learning model based on a blueprint for converting a decision tree or classifier to a neural network structure.
11. 11 . A method, comprising: obtaining a plurality of machine learning models; obtaining a plurality of training data sets corresponding to the plurality of machine learning models; applying the plurality of machine learning models to at least a portion of the plurality of training data sets to obtain output sets associated with the plurality of machine learning models; generating, based on whether the plurality of machine learning models are of a same model type, a singular machine learning model by first combining one or more models, at an output layer, before generating the singular machine learning model, wherein the singular machine learning model overfits the plurality of machine learning models; training the singular machine learning model using at least another portion of the corresponding training data sets, wherein the trained singular machine learning model is executable locally on a client device without needing to transmit inputs to a remote server to obtain results; and outputting the trained singular machine learning model by transmitting the trained singular machine learning model to the client device.
12. 12 . The method of claim 11 , further comprising: applying one or more weights to the outputs during mapping.
13. 13 . The method of claim 12 , wherein the one or more weights are equal to each other.
14. 14 . The method of claim 12 , wherein the one or more weights comprise inputs from a user.
15. 15 . The method of claim 11 , further comprising: extracting one or more feature vectors, used to generate the singular machine learning model, from the plurality of training data sets.
16. 16 . The method of claim 11 , further comprising: training the singular machine learning model using at least one new training data set.
17. 17 . The method of claim 11 , wherein training the singular machine learning model comprises recursive adjustments of one or more parameters of the singular machine learning model.
18. 18 . The method of claim 15 , wherein the one or more feature vectors are determinative, at least in part, of one or more of the outputs.
19. 19 . The method of claim 11 , wherein the plurality of machine learning models includes an ensemble model that combines the one or more models, and wherein the one or more models are combined before the plurality of machine learning models are applied.
20. 20 . A method, comprising: obtaining, from an input recognizer, a training data set; applying a plurality of machine learning models to at least a portion of the training data set to obtain outputs associated with the models; generating a singular machine learning model based on combining one or more models, at an output layer, before generating the singular machine learning model; applying the singular machine learning model to the training data set to obtain an output; adjusting one or more parameters of the singular machine learning model based on comparing the outputs to the output of the singular machine learning model.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 16/162,639, filed on Oct. 17, 2018. The contents of the above-referenced application is incorporated herein by reference in its entirety. TECHNICAL FIELD The present disclosure relates generally to the field of machine learning algorithms. More specifically, and without limitation, this disclosure relates to systems and methods for generating singular ensemble machine learning models. BACKGROUND Machine learning models, such as linear regression, neural networks, and the like have become more prevalent for predictive modeling in recent years. Some uses of predictive modeling, such as providing suggested inputs to a user of a smartphone, a tablet, or the like, are more efficient if the machine learning model is executed directly on the user device rather than on a remote server. However, individual machine learning models suffer from inaccuracies. Accordingly, some predictive systems use models that are ensembles of individual models. Such known techniques for ensemble modeling suffer from multiple drawbacks, however. For example, such ensemble models require increased processing power and memory resources, leaving them unsuitable for execution directly on the user device. Moreover, such ensemble models may require a combination of results from disparate model types (e.g., combining output of a neural network and output of a linear regression). This may require different combinatory functions because each model type may produce different output (e.g., a single prediction, a plurality of predictions with corresponding confidence levels, or the like). A need, therefore, exists for systems and methods providing ensemble models with greater efficiency than extant ensemble models. Moreover, a need exists for systems and models providing ensemble models that handle input models of different types. The disclosed systems and methods provide technological solutions to at least these existing problems. SUMMARY Embodiments of the present disclosure provide for singular ensemble models. An ensemble model, as disclosed herein, represents a singular model rather than a combinatory function applied to the outputs of a plurality of models. In this manner, the disclosed embodiments can provide a marked improvement over inefficient extant processes, as well as handle ensembles of a plurality of different model types. In one embodiment, a system for generating a singular ensemble model may comprise at least one processor and at least one storage medium storing instructions that, when executed, configure the processor to perform operations. The operations may comprise obtaining a plurality of machine learning models; obtaining a training data set; applying the plurality of machine learning models to the training data set to obtain outputs associated with the models; mapping the outputs to features of the models; combining the mapped features of the models into a singular machine learning model; training the singular machine learning model using the training data set; and outputting the trained singular machine learning model. In one embodiment, a system for generating a singular ensemble model may comprise at least one processor and at least one storage medium storing instructions that, when executed, configure the processor to perform operations. The operations may comprise obtaining a plurality of machine learning models; obtaining a plurality of training data sets, each set corresponding to one of the machine learning models; applying the plurality of machine learning models to the corresponding training data sets to obtain output sets associated with the models, each output set corresponding to one of the machine learning models; combining the output sets to form a final output set; mapping the final output set to features of the models; combining the mapped features of the models into a singular machine learning model; training the singular machine learning model using the corresponding training data sets; and outputting the trained singular machine learning model. In one embodiment, a system for generating a singular ensemble model may comprise at least one processor and at least one storage medium storing instructions that, when executed, configure the processor to perform operations. The operations may comprise obtaining a plurality of machine learning models; obtaining a training data set; applying the plurality of machine learning models to the training data set to obtain outputs associated with the models; mapping the outputs to features of the models; combining the mapped features of the models into a singular machine learning model; applying the singular machine learning model to the training data set to obtain output; comparing the outputs to the output of the singular machine learning model; adjusting one or more parameters of the singular machine learning model based on the comparison; and outputting th