US-12626788-B2 - Incrementally training a knowledge graph embedding model from biomedical knowledge graphs

Abstract

A device may train a KGE model based on an initial knowledge graph, and may generate an initial embedding matrix based on training the KGE model. The device may receive a new KG representing new information, and may convert the new KG to new KG triples. The device may generate corruption data based on the new KG triples, and may generate embeddings based on the new embedding matrix and the corruption data. The device may process the embeddings, with the KGE model, to generate scores and a loss, and may regularize the embeddings and the scores/loss for seen and unseen concepts. The device may calculate a regularized loss based on regularizing the embeddings, the scores, and the loss, and may calculate an incremental learning loss based on the loss and the regularized loss. The device may train the KGE model based on the scores and the incremental learning loss.

Inventors

• Sumit Pai
• Luca Costabello

Assignees

• ACCENTURE GLOBAL SOLUTIONS LIMITED

Dates

Publication Date

20260512

Application Date

20221018

Claims (20)

1. 1 . A method, comprising: training, by a device, a knowledge graph embedding (KGE) model based on an initial knowledge graph representing information; generating, by the device, an initial embedding matrix based on training the KGE model, wherein the KGE model includes an input layer, a corruption generation layer, an embedding lookup layer, a scoring layer and a loss layer; receiving, by the device, a new knowledge graph representing new information, with the input layer of the KGE model; converting, by the device, the new knowledge graph to a new embedding matrix comprising new knowledge graph triples corresponding to the new knowledge graph, with the input layer of the KGE model; generating, by the device, corruption triples based on the new knowledge graph triples, of the new embedding matrix, and with the corruption generation layer of the KGE model; generating, by the device, embeddings based on the new knowledge graph triples and the corruption triples and with the embedding lookup layer of the KGE model; processing, by the device, the embeddings of the embedding look up layer, with the KGE model, to generate scores with the scoring layer and a loss with the loss layer, wherein processing the embeddings of the embedding look up layer comprises: applying, by the device, a plausibility score to each of the new knowledge graph triples, and the corresponding corruption triples, with the scoring layer of the KGE model; and determining, by the device, the loss based on the plausibility scores, with the loss layer of the KGE model; regularizing, by the device, the embeddings for seen and unseen concepts; calculating providing, by the device, a regularized loss based on regularizing the embeddings; providing, by the device, an incremental learning loss based on the loss of the loss layer and the regularized loss; and training, by the device, the KGE model based on the scores of the scoring layer and the incremental learning loss by incrementally learning embeddings of the unseen concepts, and updating embeddings of the seen concepts to generate a trained KGE model.
2. 2 . The method of claim 1 , further comprising: merging the initial embedding matrix and the new embedding matrix to generate a final embedding matrix for the trained KGE model.
3. 3 . The method of claim 1 , further comprising: converting the initial knowledge graph to the initial embedding matrix with the input layer of the KGE model; generating initial corruption data based on initial knowledge graph triples and with the corruption generation layer of the KGE model; generating initial embeddings based on the initial knowledge graph triples and initial corruption triples and with an embedding lookup layer of the KGE model; and processing the initial embeddings, with the KGE model, to compute a score and a loss and to generate a trained KGE model where the initial embeddings are trained based on the initial knowledge graph.
4. 4 . The method of claim 1 , wherein generating the corruption triples based on the new knowledge graph triples comprises: generating the corruption triples by randomly replacing subjects or objects of the new knowledge graph triples with random entities from the new knowledge graph.
5. 5 . The method of claim 1 , wherein processing the embeddings, with the KGE model, to generate the scores comprises: processing the embeddings, with a scoring function, to calculate the scores, wherein the scoring function includes one or more of: a TransE scoring function, a DistMult scoring function, a ComplEx scoring function, or a HoIE scoring function.
6. 6 . The method of claim 1 , wherein training the KGE model based on the scores and the incremental learning loss to generate the trained KGE model comprises: minimizing the incremental learning loss based on the scores; and generating the trained KGE model based on minimizing the incremental learning loss.
7. 7 . The method of claim 1 , wherein regularizing the embeddings and the loss for seen and unseen concepts comprises: applying a first weight to the seen concepts; applying a second weight to the unseen concepts; and regularizing the embeddings and calculating the regularized loss based on applying the first weight to the seen concepts and applying the second weight to the unseen concepts.
8. 8 . A device, comprising: one or more memories; and one or more processors, coupled to the one or more memories, configured to: train a knowledge graph embedding (KGE) model based on an initial knowledge graph representing information; generate an initial embedding matrix based on training the KGE model, wherein the KGE model includes an input layer, a corruption generation layer, an embedding lookup layer, a scoring layer and a loss layer; receive a new knowledge graph representing new information, with the input layer of the KGE model; convert the new knowledge graph to a new embedding matrix with the input layer of the KGE model, wherein the new embedding matrix comprising new knowledge triples corresponding to the new knowledge graph; generate corruption triples based on the new knowledge graph triples, of the new embedding matrix, and with the corruption generation layer of the KGE model; generate embeddings based on the new knowledge graph triples and the corruption triples and with the embedding lookup layer of the KGE model process the embeddings, with the KGE model, to generate scores with the scoring layer and a loss with the loss layer, wherein to process the embeddings the one or more processors are configured to: apply a plausibility score to each of the new knowledge graph triples, and the corresponding corruption triples, with the scoring layer of the KGE model; and determine the loss based on the plausibility scores, with the loss layer of the KGE model; regularize the embeddings of the embedding lookup layer for seen and unseen concepts; provide a regularized loss based on regularizing the embeddings; provide an incremental learning loss based on the loss of the loss layer and the regularized loss; and train the KGE model based on the scores of the scoring layer and the incremental learning loss by incrementally training embeddings of unseen concepts, and updating embeddings of the seen concepts to generate a trained KGE model.
9. 9 . The device of claim 8 , wherein the one or more processors are further configured to: merge the initial embedding matrix and the new embedding matrix to generate a final embedding matrix for the trained KGE model.
10. 10 . The device of claim 8 , wherein the one or more processors are further configured to: convert the initial knowledge graph to the initial embedding matrix with the input layer of the KGE model; generate initial corruption data based on initial knowledge graph triples and with the corruption generation layer of the KGE model; generate initial embeddings based on the initial knowledge graph triples and initial corruption triples and with an embedding lookup layer of the KGE model; and process the initial embeddings, with the KGE model, to compute a score and a loss and to generate a trained KGE model where the initial embeddings are trained based on the initial knowledge graph.
11. 11 . The device of claim 8 , wherein the one or more processors, to generate the corruption triples based on the new knowledge graph triples, are configured to: generate the corruption triples by randomly replacing subjects or objects of the new knowledge graph triples with random entities from the new knowledge graph.
12. 12 . The device of claim 8 , wherein the one or more processors, to process the embeddings, with the KGE model, to generate the scores, are configured to: process the embeddings, with a scoring function, to calculate the scores, wherein the scoring function includes one or more of: a TransE scoring function, a DistMult scoring function, a ComplEx scoring function, or a HoIE scoring function.
13. 13 . The device of claim 8 , wherein the one or more processors, to train the KGE model based on the scores and the incremental learning loss to generate the trained KGE model, are configured to: minimize the incremental learning loss based on the scores; and generate the trained KGE model based on minimizing the incremental learning loss.
14. 14 . The device of claim 8 , wherein the one or more processors, to regularize the embeddings and the loss for seen and unseen concepts, are configured to: apply a first weight to the seen concepts; apply a second weight to the unseen concepts; and regularize the embeddings and calculating the regularized loss based on applying the first weight to the seen concepts and applying the second weight to the unseen concepts.
15. 15 . A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the device to: train a knowledge graph embedding (KGE) model based on an initial knowledge graph representing information; generate an initial embedding matrix based on training the KGE model, wherein the KGE model includes an input layer, a corruption generation layer, an embedding lookup layer, a scoring layer and a loss layer; receive a new knowledge graph representing new information, with the input layer of the KGE model; convert the new knowledge graph to a new embedding matrix with an input layer of the KGE model, wherein the new embedding matrix comprising new knowledge triples corresponding to a new knowledge graph; generate corruption triples based on the new knowledge graph triples, of the new embedding matrix, and with the corruption generation layer of the KGE model; generate embeddings based on the new knowledge graph triples and corruption triples and with the embedding lookup layer of the KGE model; process the embeddings, with the KGE model, to generate scores with the scoring layer and a loss with the loss layer, wherein to process the embeddings the one or more processors are configured to: apply a plausibility score to each of the new knowledge graph triples, and the corresponding corruption triples, with the scoring layer of the KGE model; and determine the loss based on the plausibility scores, with the loss layer of the KGE model; regularize the embeddings of the embedding lookup layer for seen and unseen concepts; provide a regularized loss based on regularizing the embeddings; provide an incremental learning loss based on the loss of the loss layer and the regularized loss; and train the KGE model based on the scores of the scoring layer and the incremental learning loss by incrementally training embeddings of unseen concepts, and updating embeddings of the seen concepts to generate a trained KGE model.
16. 16 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions further cause the device to: merge the initial embedding matrix and the new embedding matrix to generate a final embedding matrix for the trained KGE model.
17. 17 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions further cause the device to: convert the initial knowledge graph to the initial embedding matrix with the input layer of the KGE model; generate initial corruption data based on initial knowledge graph triples and with the corruption generation layer of the KGE model; generate initial embeddings based on the initial knowledge graph triples and initial corruption triples and with an embedding lookup layer of the KGE model; and process the initial embeddings, with the KGE model, to compute a score and a loss and to generate a trained KGE model where the initial embeddings are trained based on the initial knowledge graph.
18. 18 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to generate the corruption triples based on the new knowledge graph triples, cause the device to: generate the corruption triples by randomly replacing subjects or objects of the new knowledge graph triples with random entities from the new knowledge graph.
19. 19 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to process the embeddings, with the KGE model, to generate the scores, cause the device to: process the embeddings, with a scoring function, to calculate the scores, wherein the scoring function includes one or more of: a TransE scoring function, a DistMult scoring function, a ComplEx scoring function, or a HoIE scoring function.
20. 20 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to train the KGE model based on the scores and the incremental learning loss to generate the trained KGE model, cause the device to: minimize the incremental learning loss based on the scores; and generate the trained KGE model based on minimizing the incremental learning loss.

Description

BACKGROUND Current methods of new drug discovery are time consuming and expensive. Machine learning may be utilized to discover new drugs. Machine learning is a type of artificial intelligence that allows software applications to become more accurate at predicting outcomes without being explicitly programmed. SUMMARY Some implementations described herein relate to a method. The method may include training a knowledge graph embedding (KGE) model based on an initial knowledge graph representing information, and generating an initial embedding matrix based on training the KGE model. The method may include receiving a new knowledge graph representing new information, and converting the new knowledge graph to a new embedding matrix with an input layer of the KGE model. The method may include generating corruption triples based on new knowledge graph triples, of the new embedding matrix, and with a corruption generation layer of the KGE model, and generating embeddings based on the new knowledge graph triples and the corruption triples and with an embedding lookup layer of the KGE model. The method may include processing the embeddings, with the KGE model, to generate scores and a loss, and regularizing the embeddings for seen and unseen concepts. The method may include calculating a regularized loss based on regularizing the embeddings, and calculating an incremental learning loss based on the loss and the regularized loss. The method may include training the KGE model based on the scores and the incremental learning loss to generate a trained KGE model. Some implementations described herein relate to a device. The device may include one or more memories and one or more processors coupled to the one or more memories. The one or more processors may be configured to train a KGE model based on an initial knowledge graph representing information, and generate an initial embedding matrix based on training the KGE model. The one or more processors may be configured to receive a new knowledge graph representing new information, and convert the new knowledge graph to a new embedding matrix with an input layer of the KGE model. The one or more processors may be configured to generate corruption triples based on new knowledge graph triples, of the new embedding matrix, and with a corruption generation layer of the KGE model, and generate embeddings based on the new knowledge graph triples and the corruption triples and with an embedding lookup layer of the KGE model. The one or more processors may be configured to process the embeddings, with the KGE model, to generate scores and a loss, and regularize the embeddings for seen and unseen concepts. The one or more processors may be configured to calculate a regularized loss based on regularizing the embeddings, and calculate an incremental learning loss based on the loss and the regularized loss. The one or more processors may be configured to train the KGE model based on the scores and the incremental learning loss to generate a trained KGE model. Some implementations described herein relate to a non-transitory computer-readable medium that stores a set of instructions for a device. The set of instructions, when executed by one or more processors of the device, may cause the device to train a KGE model based on an initial knowledge graph representing information, and generate an initial embedding matrix based on training the KGE model. The set of instructions, when executed by one or more processors of the device, may cause the device to receive a new knowledge graph representing new information, and convert the new knowledge graph to a new embedding matrix with an input layer of the KGE model. The set of instructions, when executed by one or more processors of the device, may cause the device to generate corruption triples based on new knowledge graph triples, of the new embedding matrix, and with a corruption generation layer of the KGE model, and generate embeddings based on the new knowledge graph triples and the corruption triples and with an embedding lookup layer of the KGE model. The set of instructions, when executed by one or more processors of the device, may cause the device to process the embeddings, with the KGE model, to generate scores and a loss, and regularize the embeddings for seen and unseen concepts. The set of instructions, when executed by one or more processors of the device, may cause the device to calculate a regularized loss based on regularizing the embeddings, and calculate an incremental learning loss based on the loss and the regularized loss. The set of instructions, when executed by one or more processors of the device, may cause the device to train the KGE model based on the scores and the incremental learning loss to generate a trained KGE model. BRIEF DESCRIPTION OF THE DRAWINGS FIGS. 1A-1K are diagrams of an example implementation described herein. FIG. 2 is a diagram of an example environment in which systems and/or methods described