US-12619002-B2 - Automated artificial intelligence model generation, training, and testing

US12619002B2US 12619002 B2US12619002 B2US 12619002B2US-12619002-B2

Abstract

Mechanisms are provided to automatically generate a machine learning (ML) computer model. The mechanisms automatically generate a plurality of aggregated dataset groups, each having original dataset(s) grouped together based on a degree of correlation between characteristics of each of the original datasets. The mechanisms automatically generate, for each aggregated dataset group, a plurality of ML computer model instances, each being a ML computer model configured with a different combination of thresholds and hyperparameters than other ML computer model instances. The plurality of ML computer model instances are executed to generate performance metric information for each ML computer model instance. The performance metric information is analyzed to select a set of ML computer model instances for the aggregated dataset. The mechanisms select one or more ML computer model instances from across all of the sets of ML computer model instances as a candidate for deployment to a decision support computing system.

Inventors

Estepan Meliksetian
Harini Srinivasan
Kewen Gu
Zhangziman Song
Rosha Pokharel

Assignees

INTERNATIONAL BUSINESS MACHINES CORPORATION

Dates

Publication Date: 20260505
Application Date: 20210922

Claims (20)

1 . A method, in a data processing system, for automatically generating a machine learning (ML) computer model, the method comprising: automatically generating a plurality of aggregated dataset groups, wherein each aggregated dataset group comprises one or more original datasets, of a plurality of original datasets, grouped together based on a predetermined quantity of weather events in the one or more original datasets to generate an aggregated dataset, wherein the plurality of original datasets comprise weather data and power outage data; automatically generating, for each aggregated dataset group in the plurality of aggregated dataset groups, a plurality of ML computer model instances, wherein each ML computer model instance in the plurality of ML computer model instances for the aggregated dataset group is generated by configuring a ML computer model with a different combination of thresholds and hyperparameters than other ML computer model instances in the plurality of ML computer model instances for the aggregated dataset group; executing the plurality of ML computer model instances, for each aggregated dataset group, to generate predictions of power outages due to weather conditions and performance metric information for each ML computer model instance; evaluating the performance metric information for each ML computer model instance to select a set of ML computer model instances from the plurality of ML computer model instances for the aggregated dataset such that each aggregated dataset has an associated set of ML computer model instances; and selecting one or more ML computer model instances from across all of the sets of ML computer model instances as a candidate for deployment to a decision support computing system, wherein the one or more ML computer model instances are configured to generate one or more region of interest power outage predictions as a basis for one or more operations comprising: sending one or more requests for resource allocations to prepare for power outages, sending one or more requests to lower power utilization, or sending one or more notifications to authorities, residents or businesses to prepare for power outages.
2 . The method of claim 1 , wherein automatically generating a plurality of aggregated dataset groups comprises generating a pairwise correlation matrix data structure having entries corresponding to pairings of original datasets in the plurality of original datasets and specifies a corresponding degree of correlation between one or more features of the original datasets generated from raw data of the original datasets in each pairing.
3 . The method of claim 2 , wherein automatically generating a plurality of aggregated dataset groups further comprises, for each pairing, aggregating degrees of correlation across the one or more features to generate a single degree of correlation between the original datasets in the pairing to generate an aggregate correlation matrix data structure, and performing clustering of the original datasets based on the degrees of correlation specified in the aggregate correlation matrix data structure.
4 . The method of claim 1 , wherein automatically generating a plurality of ML computer model instances comprises, for each aggregated dataset group, performing machine learning training of one or more corresponding ML computer model instances based on an aggregated dataset corresponding to the aggregated dataset group to thereby generate one or more trained ML computer model instances for the aggregated dataset group.
5 . The method of claim 4 , wherein each ML computer model instance is a Docker image comprising the aggregated dataset and a trained ML computer model.
6 . The method of claim 5 , wherein each aggregated dataset group comprises a Docker container for a plurality of Docker images corresponding to the plurality of ML computer model instances.
7 . The method of claim 1 , wherein: the ML computer model is a weather based power outage prediction ML computer model that is configured to predict power outages in geographical regions due to weather events, the plurality of original datasets comprise historical weather data for a set of geographical regions, each original dataset in the plurality of original datasets being associated with a different geographical region in the set of geographical regions, and the aggregated dataset groups correspond to aggregated geographical regions, wherein at least one aggregated geographical region comprises a plurality of geographical regions in the set of geographical regions.
8 . The method of claim 7 , wherein the historical weather data comprises historical weather characteristics comprising precipitation data, snowfall data, ice accumulation data, wind speed data, wind gust data, and temperature data for a corresponding geographical region, and wherein a degree of correlation between an original dataset corresponding to the corresponding geographical region and other original datasets in the plurality of original datasets comprises calculations, for each historical weather characteristic, of a degree of correlation between that historical weather characteristic for the corresponding geographical region and a same historical weather characteristic of original datasets corresponding to each other geographical region in the set of geographical regions.
9 . The method of claim 7 , wherein a degree of correlation is determined based on geographical distance between geographical regions and correlations between historical weather data for geographical regions.
10 . The method of claim 1 , wherein the hyperparameters are data values defining parameters of a machine learning algorithm employed by the ML computer model instance, and wherein the thresholds are data values derived from statistical analysis of historical trends of input features, used to generate input features to the ML computer model instance.
11 . A computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: automatically generate a plurality of aggregated dataset groups, wherein each aggregated dataset group comprises one or more original datasets, of a plurality of original datasets, grouped together based on a predetermined quantity of weather events in the one or more original data sets to generate an aggregated dataset, wherein the plurality of original datasets comprise weather data and power outage data; automatically generate, for each aggregated dataset group in the plurality of aggregated dataset groups, a plurality of ML computer model instances, wherein each ML computer model instance in the plurality of ML computer model instances for the aggregated dataset group is generated by configuring a ML computer model with a different combination of thresholds and hyperparameters than other ML computer model instances in the plurality of ML computer model instances for the aggregated dataset group; execute the plurality of ML computer model instances, for each aggregated dataset group, to generate predictions of power outages due to weather conditions and performance metric information for each ML computer model instance; evaluate the performance metric information for each ML computer model instance to select a set of ML computer model instances from the plurality of ML computer model instances for the aggregated dataset such that each aggregated dataset has an associated set of ML computer model instances; and select one or more ML computer model instances from across all of the sets of ML computer model instances as a candidate for deployment to a decision support computing system, wherein the one or more ML computer model instances are configured to generate one or more region of interest power outage predictions as a basis for one or more operations comprising: sending one or more requests for resource allocations to prepare for power outages, sending one or more requests to lower power utilization, or sending one or more notifications to authorities, residents or businesses to prepare for power outages.
12 . The computer program product of claim 11 , wherein automatically generating a plurality of aggregated dataset groups comprises generating a pairwise correlation matrix data structure having entries corresponding to pairings of original datasets in the plurality of original datasets and specifies a corresponding degree of correlation between one or more features of the original datasets generated from raw data of the original datasets in each pairing.
13 . The computer program product of claim 12 , wherein automatically generating a plurality of aggregated dataset groups further comprises, for each pairing, aggregating degrees of correlation across the one or more features to generate a single degree of correlation between the original datasets in the pairing to generate an aggregate correlation matrix data structure, and performing clustering of the original datasets based on the degrees of correlation specified in the aggregate correlation matrix data structure.
14 . The computer program product of claim 11 , wherein automatically generating a plurality of ML computer model instances comprises, for each aggregated dataset group, performing machine learning training of one or more corresponding ML computer model instances based on an aggregated dataset corresponding to the aggregated dataset group to thereby generate one or more trained ML computer model instances for the aggregated dataset group.
15 . The computer program product of claim 14 , wherein each ML computer model instance is a Docker image comprising the aggregated dataset and a trained ML computer model.
16 . The computer program product of claim 15 , wherein each aggregated dataset group comprises a Docker container for a plurality of Docker images corresponding to the plurality of ML computer model instances.
17 . The computer program product of claim 11 , wherein: the ML computer model is a weather based power outage prediction ML computer model that is configured to predict power outages in geographical regions due to weather events, the plurality of original datasets comprise historical weather data for a set of geographical regions, each original dataset in the plurality of original datasets being associated with a different geographical region in the set of geographical regions, and the aggregated dataset groups correspond to aggregated geographical regions, wherein at least one aggregated geographical region comprises a plurality of geographical regions in the set of geographical regions.
18 . The computer program product of claim 17 , wherein the historical weather data comprises historical weather characteristics comprising precipitation data, snowfall data, ice accumulation data, wind speed data, wind gust data, temperature data, and power outage counts for a corresponding geographical region, and wherein a degree of correlation between an original dataset corresponding to the corresponding geographical region and other original datasets in the plurality of original datasets comprises calculations, for each historical weather characteristic, of a degree of correlation between that historical weather characteristic for the corresponding geographical region and a same historical weather characteristic of original datasets corresponding to each other geographical region in the set of geographical regions.
19 . The computer program product of claim 11 , wherein the hyperparameters are data values defining parameters of a machine learning algorithm employed by the ML computer model instance, and wherein the thresholds are data values derived from statistical analysis of historical trends of input features, used to generate input features to the ML computer model instance.
20 . An apparatus comprising: at least one processor; and at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to: automatically generate a plurality of aggregated dataset groups, wherein each aggregated dataset group comprises one or more original datasets, of a plurality of original datasets, grouped together based on a predetermined quantity of weather events in the one or more original datasets to generate an aggregated dataset, wherein the plurality of original datasets comprise weather data and power outage data; automatically generate, for each aggregated dataset group, a plurality of ML computer model instances, wherein each ML computer model instance in the plurality of ML computer model instances for the aggregated dataset group is generated by configuring a ML computer model with a different combination of thresholds and hyperparameters than other ML computer model instances in the plurality of ML computer model instances for the aggregated dataset group; execute the plurality of ML computer model instances, for each aggregated dataset group in the plurality of aggregated dataset groups, to generate predictions of power outages due to weather conditions and performance metric information for each ML computer model instance; evaluate the performance metric information for each ML computer model instance to select a set of ML computer model instances from the plurality of ML computer model instances for the aggregated dataset such that each aggregated dataset has an associated set of ML computer model instances; and select one or more ML computer model instances from across all of the sets of ML computer model instances as a candidate for deployment to a decision support computing system, wherein the one or more ML computer model instances are configured to generate one or more region of interest power outage predictions as a basis for one or more operations comprising: sending one or more requests for resource allocations to prepare for power outages, sending one or more requests to lower power utilization, or sending one or more notifications to authorities, residents or businesses to prepare for power outages.

Description

BACKGROUND The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for automatically generating, training, and testing artificial intelligence models, such as weather-based artificial intelligence models. Artificial intelligence (AI) increasingly utilizes machine learning computer models to model various real-world mechanisms, such as biological mechanisms, physics based mechanisms, business and commercial mechanisms, and the like, typically for classification and/or predictive purposes. Such machine learning (ML) computer models include linear regression models, logistic regression, linear discriminant analysis, decision trees, naïve Bayes, K-nearest neighbors, learning vector quantization, support vector machines, random forest, and deep neural networks. While ML computer models provide a good tool for performing such classification and/or predictive operations, the process of generating, training, and testing such ML computer models is a very time consuming and resource consuming intensive process often requiring a large amount of manual effort requiring a lot of experimentation. SUMMARY This Summary is provided to introduce a selection of concepts in a simplified form that are further described herein in the Detailed Description. This Summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. In one illustrative embodiment, a method, in a data processing system, is provided for automatically generating a machine learning (ML) computer model. The method comprises automatically generating a plurality of aggregated dataset groups. Each aggregated dataset group comprises one or more original datasets, of a plurality of original datasets, grouped together based on a calculation of a degree of correlation between characteristics associated with each of the original datasets in the plurality of original datasets to generate an aggregated dataset. The method further comprises automatically generating, for each aggregated dataset group, a plurality of ML computer model instances. Each ML computer model instance is generated by configuring a ML computer model with a different combination of thresholds and hyperparameters than other ML computer model instances in the plurality of ML computer model instances. The method also comprises executing the plurality of ML computer model instances, for each aggregated dataset group, to generate performance metric information for each ML computer model instance. Moreover, the method comprises evaluating the performance metric information for each ML computer model instance to select a set of ML computer model instances from the plurality of ML computer model instances for the aggregated dataset such that each aggregated dataset has an associated set of ML computer model instances. In addition, the method comprises selecting one or more ML computer model instances from across all of the sets of ML computer model instances as a candidate for deployment to a decision support computing system. In other illustrative embodiments, a computer program product comprising a computer useable or readable medium having a computer readable program is provided. The computer readable program, when executed on a computing device, causes the computing device to perform various ones of, and combinations of, the operations outlined above with regard to the method illustrative embodiment. In yet another illustrative embodiment, a system/apparatus is provided. The system/apparatus may comprise one or more processors and a memory coupled to the one or more processors. The memory may comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform various ones of, and combinations of, the operations outlined above with regard to the method illustrative embodiment. These and other features and advantages of the present invention will be described in, or will become apparent to those of ordinary skill in the art in view of, the following detailed description of the example embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS The invention, as well as a preferred mode of use and further objectives and advantages thereof, will best be understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings, wherein: FIG. 1 is an example block diagram illustrating the primary operational engines or modules and a process to grouping datasets in accordance with one illustrative embodiment; FIG. 2 is an example block diagram of the primary operational components of a ML computer model selection engine for a region group in accordance with one illustrative embodiment; FIG. 3 is a flowchart outlining an example operation for grouping training datasets into aggregate training