US-12626193-B2 - Adapting a machine learning model based on a second set of training data

US12626193B2US 12626193 B2US12626193 B2US 12626193B2US-12626193-B2

Abstract

Systems and methods for adapting a first machine learning model that takes clinical data as input, based on a second set of training data. The first machine learning model having been trained on a first set of training data. The method comprises adding an adaption module to the first machine learning model, the adaption module comprising a second machine learning model, and training the second machine learning model using a second set of training data to take an output of the first machine learning model as input and provide an adjusted output.

Inventors

Erina GHOSH
Larry James Eshelman

Assignees

KONINKLIJKE PHILIPS N.V.

Dates

Publication Date: 20260512
Application Date: 20190416

Claims (15)

1 . A computer-implemented method of adapting outputs of a trained model that preserves the trained model, the method comprising: identifying a first machine learning model and a second machine learning model, the first machine learning model having been trained on a first set of training data to utilize clinical data as a first phase input and generate, from the first phase input, a first phase output comprising a first vector; generating, by a mapping module comprising a mapping model of the same type of model as the second machine learning model, an offset vector configured to be included as part of a second phase input to the second machine learning model; and accessing, by an adaption module comprising the second machine learning model the second phase input, the second phase input comprising the offset vector and the first phase output, to generate a second phase output comprising an adjusted classification; wherein the mapping model is trained using a plurality of first phase outputs and one or more of predetermined offsets or predetermined second phase outputs, and the second machine learning model is trained to generate the adjusted classification using a second set of training data and is configured to accept as input a matrix comprising the offset vector and a vector of the same type as the first phase vector.
2 . A method as in claim 1 , wherein the second set of training data relates to a patient population.
3 . A method as in claim 2 , further comprising: deploying the first machine learning model and the adaption module together for use in producing an adjusted output for the patient population.
4 . A method as in claim 1 , further comprising: repeating the adding and training to produce an additional adaption module for another patient population.
5 . A method as in claim 1 , wherein the first set of training data comprises a first set of input parameters, the second set of training data comprises one or more additional input parameters that are different to the parameters of the first set of input parameters, and wherein the training comprises: training the second machine learning model using the second set of training data to produce the adjusted output by taking the additional input parameters into account.
6 . A method as in claim 1 , wherein the second machine learning model comprises a boosting model configured to determine the adjusted output based on a summation of a plurality of classifiers.
7 . A method as in claim 6 , wherein the boosting model is further configured to: determine, from the mapping relationship, an initial offset for an unadjusted output from the second machine learning model by converting the output of the first machine learning model into the initial offset for the unadjusted output from the second machine learning model based on the mapping relationship; and apply the initial offset to the summation of the plurality of classifiers, based on the output of the first machine learning model, so as to initialize the boosting model to produce an equivalent output to the first machine learning model if data corresponding to the second set of training data is unavailable.
8 . A method as in claim 7 , further comprising: determining a mapping relationship for converting an output of the first machine learning model into an initial offset.
9 . A method as in claim 1 , wherein determining a mapping relationship comprises: providing a plurality of outputs of the first machine learning model and a plurality of respective classifications to a mapping model; and determining a mapping relationship from the output of the mapping model.
10 . A method as in claim 1 , further comprising: providing new clinical data as input to the first machine learning model; outputting a new output from the first machine learning model; providing the new output as an input to the adaption module; and outputting a new adjusted output from the adaption module.
11 . A method as in claim 10 , further comprising the adaption module: mapping the new output to an initial offset, using the determined mapping relationship; and initializing the second machine learning model, using the initial offset.
12 . A method as in claim 10 , further comprising: outputting the new output and the adjusted new output.
13 . A system for adapting outputs of a trained model that preserves the trained model, the system comprising: a memory comprising instruction data representing a set of instructions; a processor configured to communicate with the memory and to execute the set of instructions, wherein the set of instructions, when executed by the processor, cause the system to: identify a first machine learning model and a second machine learning model, the first machine learning model having been trained on a first set of training data to utilize clinical data as a first phase input and generate, from the first phase input, a first phase output comprising a first vector; generate an offset vector configured to be included as part of a second phase input to the second machine learning model; and access, with an adaption module comprising the second machine learning model, the second phase input, the second phase input comprising the offset vector and the first phase output, to generate a second phase output comprising an adjusted classification; wherein the mapping model is trained using a plurality of first phase outputs and one or more of predetermined offsets or predetermined second phase outputs, and the second machine learning model is trained to generate the adjusted classification using a second set of training data and is configured to accept as input a matrix comprising the offset vector and a vector of the same type as the first phase vector.
14 . A non-transitory computer readable medium that stores instructions, which when executed by one or more processors, cause the one or more processors to: identify a first machine learning model and a second machine learning model, the first machine learning model having been trained on a first set of training data to utilize clinical data as a first phase input and generate, from the first phase input a first phase output comprising a first vector; generate an offset vector configured to be included as part of a second phase input to the second machine learning model; and access, with an adaption module comprising the second machine learning model, the second phase input, the second phase input comprising the offset vector and the first phase output, to generate a second phase output comprising an adjusted classification; wherein the mapping model is trained using a plurality of first phase outputs and one or more of predetermined offsets or predetermined second phase outputs, and the second machine learning model is trained to generate the adjusted classification using a second set of training data and is configured to accept as input a matrix comprising the offset vector and a vector of the same type as the first phase vector.
15 . The method as in claim 2 , wherein the patient population comprises patients associated with one or more of: a hospital; a hospital system; a geographic region; and a clinical condition.

Description

CROSS-REFERENCE TO PRIOR APPLICATIONS This application is the U.S. National Phase application under 35 U.S.C. § 371 of International Application No. PCT/EP2019/059718, filed on Apr. 16, 2019, which claims the benefit of U.S. Patent Application No. 62/664,365, filed on Apr. 30, 2018. These applications are hereby incorporated by reference herein. TECHNICAL FIELD The disclosure herein relates to a system and method for adapting a machine learning model based on a second set of training data. BACKGROUND The general background is in machine learning models used in a clinical setting, e.g. used in clinical decision support systems to make clinical predictions, analyses or diagnoses. Machine learning models (such as empirical-based predictive models) may be trained on specific training data sets, using the characteristic features of the dataset. If, in use, a trained model is used to classify or process data that is not represented in the training data set used to train the model (e.g. new data from a different population compared to the population(s) used to train the model), then it may not perform as well. As such, machine learning models can generally only be used on similar populations to the training data set used to train the model. Since it is very difficult to create a training dataset which encompasses examples of all possible populations (e.g. all different disease types, types of hospitals, geographical and economical settings), such machine learning models may therefore not be appropriate for use with different patient populations e.g. for patients with different chronic conditions, being treated at different hospitals or regions with different care practices. SUMMARY As noted above, a machine learning model trained on training data related to a particular patient population may not produce accurate outputs for other patient populations. One standard approach to address this problem is to train a new model (or re-train an old one using new training data) for each population, e.g. using training data specific to that population. This essentially creates a new model for every dataset on which the model is trained. However, this approach may be resource intensive. It can also result in very different models which may cause problems, for example, if a model needs regulatory approval before it can be deployed. Different models can also make the integration of a model's interface with a workflow more difficult. Furthermore, different versions of a clinical decision support tool will ideally have similar looking outputs for similar patients. These issues may be more problematic if a model is already being used, and one wants to deploy it for patients with different conditions or features. An alternative approach to re-training a model each time is to train a new model that uses the new conditions/features, and then integrates (e.g. averaging or otherwise combining) the outputs of the two trained models. The state of the art solution for combining the outputs of two or more models is known as “stacking”. Although this approach uses the output of the original model, when combined with the output of a second, completely separate model, the combined output may bear little resemblance to that of the original model. This may erode trust in the output and the final outputs may lack transparency e.g. it may not be easy to tell how an output is changed by the stacking process. There is therefore a need for systems and methods that improve on the solutions described above to enable a machine learning model to be updated based on additional training data, in a transparent and robust way. According to a first aspect there is provided a method of adapting, based on a second set of training data, a first machine learning model that takes clinical data as input, the first machine learning model having been trained on a first set of training data. The method comprises adding an adaption module to the first machine learning model, the adaption module comprising a second machine learning model, and training the second machine learning model using the second set of training data, to take an output of the first machine learning model as input and provide an adjusted output. According to a second aspect there is provided a system for adapting, based on a second set of training data, a first machine learning model that takes clinical data as input, the first machine learning model having been trained on a first set of training data. The system comprises a memory comprising instruction data representing a set of instructions and a processor configured to communicate with the memory and to execute the set of instructions. The set of instructions, when executed by the processor, cause the processor to add an adaption module to the first machine learning model, the adaption module comprising a second machine learning model, and train the second machine learning model using the second set of training data, to take an output of the fi