US-12619885-B2 - Method for operating a neural link prediction model and a corresponding system

Abstract

A method for operating a neural link prediction model includes training a neural link predictor of the neural link prediction model using a knowledge base of facts. During the training: a fact (training triple) is processed by the neural link predictor to obtain a behavior or prediction, parameters are updated based on the behavior or the prediction, an influence indicating a change of the parameters caused by the training triple is determined, and is collected and stored. Subsequent to training, the trained neural link predictor is provided for use in maintaining the knowledge base of facts. Embodiments of the present disclosure can facilitate decision-making by using machine learning (e.g., help a user make a decision) and can be used in a variety of applications including, but not limited to, several use cases in drug development for medical/healthcare, fact checking of news, and/or product recommendation.

Inventors

• Carolin Lawrence
• Timo Sztyler
• Mathias Niepert

Assignees

• NEC Laboratories Europe GmbH

Dates

Publication Date

20260505

Application Date

20200804

Priority Date

20200604

Claims (18)

1. 1 . A computer-implemented method for operating a neural link prediction model, the method comprising: training a neural link predictor of the neural link prediction model using a knowledge base comprising a plurality of facts; during the training of the neural link predictor of the neural link prediction model: processing the plurality of facts by the neural link predictor to obtain behaviors or predictions, wherein the plurality of facts comprises a plurality of training triple, updating parameters of the neural link predictor based on the behaviors or the predictions, based on updating the parameters of the neural link predictor, determining changes of the parameters of the neural link predictor that are caused by the plurality of training triples, and collecting and storing the changes of the parameters of the neural link predictor that are caused by the plurality of training triples in a memory; and subsequent to the training of the neural link predictor, obtaining a new fact candidate; generating a candidate prediction for the new fact candidate using the trained neural link predictor of the neural link prediction model; based on the changes of the parameters of the neural link predictor that are stored in the memory and associated with the plurality of training triples, determining one or more facts from the plurality of facts of the knowledge base that reverses the generated candidate prediction for the new fact candidate when removed by: removing a change of the parameters of the neural link predictor from the changes of the parameters that are stored in the memory, wherein the change is associated with a first fact from the plurality of facts of the knowledge base; based on removing the change, determining a new candidate prediction; and determining the first fact reverses the generated candidate prediction for the new fact candidate based on the new candidate prediction; and maintaining the knowledge base of the plurality of facts based on the one or more determined facts.
2. 2 . The computer-implemented method according to claim 1 , wherein the changes of the parameters are aggregated and/or computed into an influence score between the plurality of facts that already exist in the knowledge base and the new fact candidate or into an influence score for each fact of the plurality of facts with respect to the candidate prediction with regard to the new fact candidate.
3. 3 . The computer-implemented method according to claim 2 , wherein on the basis of the influence score, the plurality of facts in the knowledge base are determined that support or contradict the candidate prediction of the neural link predictor with regard to the new fact candidate.
4. 4 . The method according to claim 1 , wherein maintaining the knowledge base comprises determining whether the new fact candidate has enough evidence or support from the plurality of facts of the knowledge base.
5. 5 . The computer-implemented method according to claim 1 , wherein maintaining the knowledge base is based on predicting a probability that the new fact candidate is correct or false.
6. 6 . The computer-implemented method according to claim 1 , wherein maintaining the knowledge base is based on predicting a probability that the new fact candidate can be added to the knowledge base as being a certain fact, wherein the probability that the new fact candidate is the certain fact is based on the one or more facts from the plurality of facts of the knowledge base that reverses the generated candidate prediction for the new fact candidate when removed.
7. 7 . The computer-implemented method according to claim 1 , wherein collecting and storing the changes in the parameters is performed during a retraining of the neural link predictor.
8. 8 . The computer-implemented method according to claim 1 , wherein a bi-partite graph between the plurality of facts and the parameters of the neural link predictor is generated.
9. 9 . The computer-implemented method according to claim 1 , wherein updating the parameters of the neural link predictor comprises performing a Stochastic Gradient Descent (SGD) to update the parameters of the neural link predictor.
10. 10 . The computer-implemented method according to claim 1 , wherein the method is applied to question answering systems or fact checking systems.
11. 11 . The computer-implemented method according to claim 10 , wherein the question answering systems are biomedical question answering systems.
12. 12 . The computer-implemented method according to claim 1 , wherein the method is applied to recommender systems.
13. 13 . The computer-implemented method of claim 1 , wherein maintaining the knowledge base of the plurality of facts comprises: including the new fact candidate in the knowledge base based on the candidate prediction.
14. 14 . The computer-implemented method of claim 13 , wherein maintaining the knowledge base of the plurality of facts further comprises: determining a candidate prediction change in the candidate prediction based on the changes of the parameters of the neural link predictor that are stored in the memory; and comparing the candidate prediction change with one or more thresholds, wherein including the new fact candidate in the knowledge base is based on the comparison.
15. 15 . The computer-implemented method of claim 13 , wherein maintaining the knowledge base of the plurality of facts further comprises: determining a number of the one or more facts from the plurality of facts of the knowledge base that reverses the generated candidate prediction for the new fact candidate when removed; and based on comparing the number with one or more thresholds, displaying the one or more facts from the plurality of facts of the knowledge base to a user, wherein including the new fact candidate in the knowledge base is based on user input after the user reviews the displayed one or more facts.
16. 16 . The computer-implemented method of claim 1 , wherein the new fact candidate is a triple comprising three elements, wherein one of the three elements of the triple is omitted.
17. 17 . The computer-implemented method of claim 1 , further comprising: determining a ranked list based on the one or more facts from the plurality of facts of the knowledge base that reverses the generated candidate prediction for the new fact candidate when removed, and wherein maintaining the knowledge base of the plurality of facts is based on the ranked list.
18. 18 . The computer-implemented method of claim 1 , wherein determining the one or more facts from the plurality of facts of the knowledge base that reverses the generated candidate prediction for the new fact candidate when removed is based on dot products associated with gradients of the candidate prediction.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a U.S. National Phase application under 35 U.S.C. § 371 of International Application No. PCT/EP2020/071923, filed on Aug. 4, 2020, and claims benefit to European Patent Application No. EP 20178215.8, filed on Jun. 4, 2019. The International Application was published in English on Dec. 9, 2021, as WO 2021/244766 A1 under PCT Article 21(2). FIELD Embodiments of the present invention relate to a method for operating a neural link prediction model, and a system for operating a neural link prediction model. BACKGROUND A neural link prediction model can be a machine learning model that predicts the probability of a new fact given the already existing facts in the knowledge base. Corresponding prior art documents are listed as follows: Moritz Hardt, Benjamin Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In Proceedings of the 33rd International Conference on International Conference on Machine Learning (ICML), page 1225-1234, 2016.Pouya Pezeshkpour, Yifan Tian, and Sameer Singh. Investigating robustness and interpretability of link prediction via adversarial modifications. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019.Koh, Pang Wei, and Percy Liang. “Understanding black-box predictions via influence functions.” In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1885-1894. JMLR. org, 2017. Estimating the influence of training examples on a machine learning model's behavior or a neural link prediction model's behavior is an important problem. For instance, it can be used for identifying training examples most responsible for a given prediction and, therefore, faithfully explain the output of a black-box model to a user. Besides providing an understanding of model behavior, it can also be used to find adversarial examples, to uncover domain mismatch, and to determine incorrect or mislabeled examples. SUMMARY In an embodiment, the present disclosure provides a method for operating a neural link prediction model. The method includes training a neural link predictor of the neural link prediction model using a knowledge base of facts, estimating an influence of at least one fact—being provided to the neural link predictor—on a behavior or prediction of the neural link predictor, collecting and storing the influence of the at least one fact on at least one parameter of the neural link predictor during training in a memory. The influence is expressed by a change in a value of the at least one parameter. BRIEF DESCRIPTION OF THE DRAWINGS Subject matter of the present disclosure will be described in even greater detail below based on the exemplary figures. All features described and/or illustrated herein can be used alone or combined in different combinations. The features and advantages of various embodiments will become apparent by reading the following detailed description with reference to the attached drawings, which illustrate the following: FIG. 1 shows in a diagram an overview of an embodiment of the proposed invention. DETAILED DESCRIPTION Embodiments of the present invention relate to maintaining a knowledge base of facts, such as “Protein X is targeted by Drug Y” or “COVID-19 has symptom Cough.” The knowledge base can be used to find documents that are factually wrong, e.g., statements in the document that contradict what is in the knowledge base—fact checking—, or can be used to answer questions about biological processes, e.g., “What drug could be useful for disease X?”. One problem is that the knowledge base should remain up to date and ingest new data while at the same time making sure that the newly added facts are correct. Embodiments of the present invention provide a method for operating a neural link prediction model, wherein a knowledge base of facts is used for training a neural link predictor of the neural link prediction model, wherein an influence of at least one fact—being provided to the neural link predictor—on a behavior or prediction of the neural link predictor is estimated, wherein the influence of the at least one fact on at least one parameter of the neural link predictor during training is collected and stored within a memory, wherein the influence is expressed by a change in a value of the at least one parameter. Embodiments of the present invention provide a system for operating a neural link prediction model. The system includes a knowledge base of facts for training a neural link predictor of the neural link prediction model, wherein an influence of at least one fact—being provided to the neural link predictor—on a behavior or prediction of the neural link predictor is estimated by an estimating means. The system further includes collecting and storing means for collecting and storing the influence of the at least one fact on at least one parameter of the neural link predic