CN-116861002-B - Causal common sense knowledge base construction method based on graph attention mechanism

CN116861002BCN 116861002 BCN116861002 BCN 116861002BCN-116861002-B

Abstract

The invention discloses a causal common sense knowledge base construction method based on a graph attention mechanism, which comprises the following steps of 1) obtaining causal common sense knowledge triples from an open knowledge base to serve as training resources, 2) obtaining vector representations of the causal common sense knowledge triples by adopting a knowledge graph embedding technology, 3) constructing a causal common sense knowledge base construction model based on the graph attention mechanism by combining with a background knowledge graph, selecting a proper loss function to optimize model parameters, and 4) scoring the confidence of the missing triples by utilizing the causal common sense knowledge base construction model, and adding the triples with the highest score as the causal common sense knowledge of new learning into the knowledge base. The invention builds a causal common sense knowledge base construction model based on the background knowledge graph by adopting a graph attention mechanism, aims to learn new causal common sense knowledge through a small amount of training resources, expands the causal common sense knowledge base, and enables the knowledge base to better support an intelligent question-answering system.

Inventors

YU KUI
LI JINDI
LI YULING
WANG YUWEI

Assignees

合肥工业大学

Dates

Publication Date: 20260508
Application Date: 20230713

Claims (4)

1. The causal common sense knowledge base construction method based on the graph attention mechanism is characterized by comprising the following steps: step one, collecting causal common sense knowledge from a knowledge base and carrying out structural representation to construct a triplet set of causal common sense knowledge , wherein, Represents a general sense of cause and effect, Representing the head entity and the position of the head entity, The cause-and-effect relationship is represented, Represents the tail entity of the plant, As a set of entities, Is a causal relation set; Step two, collecting causal common sense knowledge triplet set The triples related to the causal relationship of the medium and high frequency occurrence are stored and modeled in the form of a directed graph by utilizing priori knowledge, thereby constructing a background knowledge graph Wherein, the head entity and the tail entity of the triplet are respectively used as the background knowledge graph Is used as the background knowledge graph Edges between the intermediate nodes; step three, adopting a knowledge graph embedding method to carry out the background knowledge graph Vector representation is performed on each node and edge in the header entity node, so that vector representation of the header entity node is obtained Vector representation of tail entity nodes Vector representation of causal relationship edges ; Step four, from causal common sense knowledge triplet set The random extraction of a causal relation is marked as Selecting a causal relationship Constructing causal relation by corresponding partial triples Is a support set of (2) By using causal relationship The remaining triples build causal relationships Is a set of queries for (1) Then collect the inquiry set All tail entities in the query set are replaced by entities which are not related with the head entities of the query set, thereby constructing the negative query set ; Step five, by causal relationship Corresponding support set And a query set Composing training tasks From a set of causal relationships Selecting other causal relationships And constructing a causal relationship according to the process of the step four Is a support set of (2) And a query set Constitute test tasks ; Step six, constructing a feature vector learning module by adopting a graph attention mechanism to serve as a support set Generating a respective vector representation for each entity pair; step 6.1, extracting background knowledge graph Useful information in (1) to obtain causal relationship Corresponding first Personal neighbor information set Wherein, the method comprises the steps of, Representing causal relationships to Is a support set of (2) Middle (f) A single head entity-tail entity pair, Representing a background knowledge graph Extract the first Personal head entity The corresponding direct neighbor information is used to determine, Representing a background knowledge graph Extract of (a) Personal head entity Is the first of (2) Individual direct neighbor entities The associated neighbor information, i.e. indirect neighbor information, Representing tail entities Is provided with a direct neighbor information of (a), Representing tail entities Is a part of the indirect neighbor information; Step 6.1.1 for causal relationship Is a support set of (2) Middle (f) Head entity-tail entity pair From background knowledge graph Extract the first Personal head entity Corresponding direct neighbor information , wherein, Represent the first Personal head entity Is a direct neighbor relation of (a); Represent the first Personal head entity Is a direct neighbor entity of (a); Represent the first Personal head entity The number of direct neighbors that are associated with, Represent the first Personal head entity An associated direct neighbor causal relationship-entity pair; Step 6.1.2 from background knowledge graph Extract of (a) Personal head entity Is the first of (2) Individual direct neighbor entities Associated neighbor information, noted as indirect neighbor information , wherein, Represent the first Personal head entity Is an indirect neighbor relation of (a); Represent the first Personal head entity Is an indirect neighbor entity of (a); Represent the first Personal head entity The number of indirect neighbors that are associated with, Represent the first Personal head entity Associated first Indirect neighbor relation-entity pairs; step 6.1.3, the direct neighbor information Indirect neighbor information Common composition of Personal head entity Neighbor information set of (a) ; Step 6.1.4, obtaining a tail entity according to the process of step 6.1.1-step 6.1.3 Is (are) direct neighbor information Indirect neighbor information Neighbor information set ; Step 6.1.5, obtaining causal relationship Corresponding first Personal neighbor information set ; Step 6.2, learning cause and effect common sense triplets by the feature vector learning module Middle (f) Pairs of individual entities Is denoted as vector representation of (2) ; Step seven, for the query set Scoring the confidence level of the causal common sense knowledge triples in the model; Step 7.1 for a query set Any one of the query triples is marked as Obtaining the current causal relationship by adopting the formula (10) -formula (11) Vector representation of (a) : (10) (11) In the formulae (10) - (11), In order to perform the dot product operation, Is a causal relationship Query triples of (1) Is used in the vector representation of (a), The causal relationship is obtained through the study of (9) Is a support set of (2) Middle (f) Head entity-tail entity pair Is used in the vector representation of (a), Is the first Pairs of individual entities Representing the corresponding attention weight; step 7.2, evaluating the query set by using the metric function shown in (12) Mid-query knowledge triples Confidence score of (2) : (12) Step 7.3 for negative query set Any one of the negative query triples is marked as According to the process from step 7.1 to step 7.2, the confidence score of the negative query triplet is calculated ; Step eight, generating causal relationship by using counter propagation algorithm pair Is optimized and a loss function of formula (13) is calculated When the loss function Continuously descending until convergence, and obtaining an optimal causal common sense knowledge base construction model: (13) in the formula (13), the amino acid sequence of the compound, Is a super parameter; And step nine, inputting a plurality of candidate entities into the optimal causal common sense knowledge base construction model, calculating the confidence score of the corresponding query triplet, and taking the triplet with the highest confidence score as the new causal common sense knowledge learned by the model.
2. The causal common sense knowledge base construction method based on graph attention mechanism according to claim 1, wherein said step 6.2 is performed as follows: Step 6.2.1, utilizing the causal relationship of the causal common sense knowledge autonomous learning task currently conducted by the aid of the step (1) Modeled as the first Personal head entity node Vector representation of (a) First, the Personal tail entity node Vector representation of (a) Translation vector between : (1) Step 6.2.2, calculating causal common sense knowledge triples by adopting the formula (2) Middle (f) Personal head entity Is a direct neighbor of (2) Causal relationship with Correlation between : (2) In the formula (2), the amino acid sequence of the compound, For the connection of the vectors, For the purpose of the transposition, And For both weight matrices, As direct neighbor entities Is used in the vector representation of (a), Is a direct neighbor relation Vector representations of (a); step 6.2.3, calculating causal common sense knowledge triples by adopting the formula (3) Middle (f) Personal head entity Is of indirect neighbors of (a) Causal relationship with Correlation between : (3) In the formula (3), the amino acid sequence of the compound, And For both weight matrices, Is an indirect neighbor entity Is used in the vector representation of (a), Is an indirect neighbor relation Vector representations of (a); step 6.2.4, aggregating direct neighbor information and indirect neighbor information with different attention weights by using the formula (4) -the formula (7), and obtaining the first formula (8) Personal head entity Is an enhanced vector representation of (1) : (4) (5) (6) (7) (8) In the formulas (4) to (8), In order to activate the function, Representation will direct neighbor relation representation And direct neighbor entity representation A representation of the vectors that are connected together, Representation will indirect neighbor relation representation Indirect neighbor entity representation A representation of the vectors that are connected together, 、、 For the three weight matrices it is possible to provide, In order for the deviation to be a function of, Representing the first of aggregated direct neighbor information Personal head entity Is used for the enhancement vector representation of (a), Representing the first of aggregated indirect neighbor information Personal head entity Is used for the enhancement vector representation of (a), Is a door mechanism; Step 6.2.5, according to the procedure of step 6.2.2-step 6.2.4, obtain the first Tail entity Is an enhanced vector representation of (1) ; Step 6.2.6, adopt (9) to carry out the first step Enhancement vector representation of individual head and tail entities 、 Connected to generate causal relationship Corresponds to the first Pairs of individual entities And is used as a causal common sense knowledge triplet learned by a feature vector learning module Middle (f) Pairs of individual entities Is denoted as vector representation of (2) ; (9) In the formula (9), the amino acid sequence of the compound, , For both weight matrices, Is the deviation.
3. An electronic device comprising a memory and a processor, wherein the memory is configured to store a program for supporting the processor to perform the causal common sense knowledge base construction method of any of claims 1-2, the processor being configured to execute the program stored in the memory.
4. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor performs the steps of the causal common sense knowledge base construction method of any of claims 1-2.

Description

Causal common sense knowledge base construction method based on graph attention mechanism Technical Field The invention belongs to the technical field of natural language processing, and particularly relates to a causal common sense knowledge base construction method based on a graph attention mechanism. Background Causal common sense knowledge refers to a common cognitive basis that people accumulate in daily life. Humans can better understand and adapt to the surrounding environment by virtue of causal common sense knowledge, and make reasonable judgment and decision. For the artificial intelligence field, the causal common sense knowledge base can provide rich background knowledge and reasoning capability for the intelligent question-answering system. Such causal-based intelligent question-answering systems have potential applications in many areas, including education, medical health, and scientific research. The causal common sense knowledge can be stored and displayed by using a knowledge graph. A knowledge graph is a structured knowledge representation that graphically presents relationships (edges) between entities (nodes) to describe the links between knowledge and concepts in the real world. In the knowledge graph, causal common sense knowledge is described in the form of triples (head entities, relations, tail entities) that constitute a huge network. And the triples can be associated to form a complex knowledge graph structure. By way of triples, knowledge maps can accurately represent relationships and attributes between concepts. The triple storage form of the common sense knowledge has the characteristics of simplicity, expandability and easy understanding. It can organize and retrieve knowledge in a structured way, providing a basis for knowledge reasoning and application. Based on the constructed causal common sense knowledge base, the intelligent question-answering system can better understand the questions of the user, identify causal relationships in the questions, infer answers and provide interpretation and inference capabilities. Currently, the knowledge base is constructed by manual construction ①, which is the most traditional and basic method, by manually collecting, sorting and writing knowledge. An expert, domain expert, or knowledge worker may manually create a knowledge base by writing documents, writing rules, defining concepts, and the like. This method is applicable to small-scale knowledge bases or knowledge bases of specific fields. ② Automatic extraction, namely automatically extracting knowledge from large-scale text data by using Natural Language Processing (NLP) and information extraction technology. Such methods may use techniques of entity recognition, relationship extraction, event extraction, etc. to automatically recognize and extract knowledge. For example, information such as company names, product characteristics, event occurrence time and the like is extracted from news articles to construct a relevant knowledge base. ③ Semantic Web (Semantic Web) technology, semantic Web is a method for organizing and representing structured knowledge. It uses RDF (Resource Description Framework) as a knowledge representation language to build a knowledge base by defining entities, attributes and relationships. Semantic web technology also provides reasoning mechanisms and query languages (e.g., SPARQL) so that the knowledge base has the capabilities of logical reasoning and flexible query. ④ The machine learning method is to learn and construct a knowledge base from large-scale data by utilizing a machine learning algorithm. Such methods may use clustering, classification, topic modeling, etc. techniques to automatically discover and organize knowledge. For example, a text classification algorithm is used to identify different topics or domains from news articles and construct a corresponding knowledge base. Although these methods have achieved some success, the construction of causal knowledge bases still faces the challenge that, first, the completeness of the known causal knowledge cannot be guaranteed, and some relationships often do not have enough triples as training corpus. Causal common sense knowledge data comes from different fields and cultural backgrounds and relates to rich entities and relationships. The existing method can not fully mine the attribute characteristics and the relation modes associated with the entities in the knowledge, and ignores the neighborhood information of the entities when building the semantic structure. Therefore, the semantic information of the obtained entity pairs may be incomplete, especially when the triad data of a certain causal relation is sparse, the vector representation of the learned causal relation is not necessarily reliable due to the fact that the neighbor information is not fully utilized, so that the prediction capability of the causal common sense knowledge base building model is limited. Second, existing methods ty