Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning

Reasoning is essential for the development of large knowledge graphs, especially for completion, which aims to infer new triples based on existing ones. Both rules and embeddings can be used for knowledge graph reasoning and they have their own advantages and difficulties. Rule-based reasoning is accurate and explainable but rule learning with searching over the graph always suffers from efficiency due to huge search space. Embedding-based reasoning is more scalable and efficient as the reasoning is conducted via computation between embeddings, but it has difficulty learning good representations for sparse entities because a good embedding relies heavily on data richness. Based on this observation, in this paper we explore how embedding and rule learning can be combined together and complement each other's difficulties with their advantages. We propose a novel framework IterE iteratively learning embeddings and rules, in which rules are learned from embeddings with proper pruning strategy and embeddings are learned from existing triples and new triples inferred by rules. Evaluations on embedding qualities of IterE show that rules help improve the quality of sparse entity embeddings and their link prediction results. We also evaluate the efficiency of rule learning and quality of rules from IterE compared with AMIE+, showing that IterE is capable of generating high quality rules more efficiently. Experiments show that iteratively learning embeddings and rules benefit each other during learning and prediction.


INTRODUCTION
Many Knowledge Graphs (KGs), such as Freebase [2] and YAGO [33], have been built in recent years and led to a broad range of applications, including question answering [4], relation extraction [36], and recommender system [49].KGs store facts as triples in the form of (subject entity, relation, object entity), abridged as (s, r, o).Some KGs also have an ontology with class and property expression axioms which place constraints on classes and types of relationships.
Knowledge graph reasoning (KGR) can infer new knowledge based on existing ones and check knowledge consistency.It is attracting research interest and is important for completing and cleaning up KGs.Two of the most common learning methods for KGR are embedding-based reasoning and rule-based reasoning [26].
One of the crucial tasks for embedding-based and rule-based reasoning is to learn embeddings and rules respectively.Embedding learning methods such as TransE [3], HolE [27] and ComplEx [35] learn latent representations of entities and relations in continuous vector spaces, called embeddings, so as to preserve the information and semantics in KGs.Embedding-based reasoning is more efficient when there are a large number of relations or triples to reason over.Rule learning methods such as AMIE [10] aim to learn deductive and interpretable inference rules.Rule-based reasoning is precise and can provide insights for inference results.
With different advantages, both embedding and rule learning are important for KGR, while they still have their own difficulties and weaknesses.
Sparsity Problem for Embedding Learning.One of the main difficulties for embedding learning is the poor capability of encoding sparse entities.For example, Figure 1 shows correlation between entity frequency and entity link prediction results measured in mean reciprocal rank (MRR), where higher values means better results.In Figure 1, the blue line shows there are a large portion of entities having only a few triples, revealing the common existence of sparse entity.The yellow line shows that the prediction results of entities are highly related to their frequency, and the results of sparse entities are much worse than those of frequent ones.Efficiency Problem for Rule Learning.The main difficulty in rule learning is the huge search space when determining rule structures and searching for support triples.For example, with a small KG containing 10 relations and 100 entities, the number of possible structures for a rule with 3 relations is 10 3 and the maximum number of supports for these rules is 100 2 * 3 * 10 3 = 10 15 .Since the search space is exponential to the number of relations, it will be much larger for real KGs than this example.
With different advantages and difficulties, we argue that embedding learning and rule learning can benefit and complement each other.On the one hand, deductive rules can infer additional triples for sparse entities and help embedding learning methods encode them better.On the other hand, embeddings encoded with rich semantics can turn rule learning from discrete graph search into vector space calculation, so that reduce the search space significantly.Thus we raise the research question: whether it is possible to learn embeddings and rules at the same time and make their advantages complement to each other's difficulties.
In this paper, we propose a novel framework IterE that iteratively learns embeddings and rules, which can combine many embedding methods and different kinds of rules.Especially, we consider linear map assumption (Section 2.1) for embedding learning because it is inherently friendly for rule learning as there are special rule conclusions for relation embeddings in rules (Table 2).We also consider a particular collection of object property axioms defined in OWL2 (Table 1) for rule learning considering that semantics included in web ontology language are important for the development of knowledge graph.
IterE mainly includes three parts: (1) embedding learning, (2) axiom induction, and (3) axiom injection.Embedding learning learns embeddings for entities and relations, with input including triples existing in KG and those inferred by rules.Axiom induction first generates a pool of possible axioms with an effective pruning strategy proposed in this paper and then assigns a score to each axiom in the pool based on calculation between relation embeddings according to rule conclusions from linear map assumption.Axiom injection utilizes axioms' deductive capability to infer new triples about sparse entities according to axiom groundings and injects these new triples into KG to improve sparse entity embeddings.These three parts are conducted iteratively during training.
We evaluate IterE from three perspectives, 1) whether axiom improves sparse embeddings' quality and their predictions, 2) whether embedding helps improve rule learning efficiency and quality, and 3) how iterative training improves both embedding and rule learning during training.The experiment results show that IterE achieves both better link prediction performance and high quality rule learning results.These support our goal of making IterE complement the strengths of embedding and rule learning.
Contributions of our work are as follows: • We propose an iterative framework that combines embedding learning and rule learning to explore the mutual benefits between them.Experiments show that it leads to better link prediction results using rules and embeddings together.

PRELIMINARIES 2.1 Knowledge graph embedding
A KG K = {E, R, T } contains a set of entities E, a set of relations R and a set of triples T = {(s, r , o)|s, o ∈ E; r ∈ R}.In a triple (s, r , o), the symbols s, r , and o denote subject entity, relation, and object entity respectively.An example of such triple is (Tokyo, locatedIn, Japan).Knowledge graph embedding (KGE) aims to embed all entities and relations in a continuous vector space, usually as vectors or matrices called embeddinдs.Embeddings can be used to estimate the likelihood of a triple to be true via a score function f : E × R × E → R. Concrete score functions are defined based on different vector space assumptions.We now describe two vector space assumptions commonly used in KGEs and their corresponding score functions.
(a) Translation-based assumption embeds entities and relations as vectors and assumes v s + v r = v o , in which v s , v r and v o are vector embeddings for s, r and o respectively.For a true triple, the relation-specific translation of subject embedding (v s + v r ) is close to the object embedding v o in embeddings' vector space.
(b) Linear map assumption embeds entities as vectors and relations as matrices.It assumes that the subject entity embedding v s can be linearly mapped to object entity embedding v o via relation embedding M r .In this case, for a true triple, the linear mapping of the subject embedding by the relation matrix (v s M r ) is close to the object embedding v o in embeddings' vector space.Thus the score function ϕ of two assumptions can be written as: where sim(x, y) calculates the similarity between vector x and y.
From the assumption point of view, v s M r = v o (or v s + v r = v o ) should exactly hold for true triples in the linear-map assumption (or translation-based assumption).From the modeling point of view, this is the optimization goal during learning, namely to be as close as possible to the equation in assumption.The learning process is done through either maximizing the objective or minimizing the error induced from assumptions given by their respective loss functions.Hence, the assumption equation usually does not exactly hold with learned embeddings, but their loss functions are designed to approach the assumption as much as possible.
In this paper, we adopt linear map assumption for embedding learning because many reasonable rule conclusions can be derived with relation embeddings based on this assumption (Table 2).

Rules Learning
Suppose X is a countable set of variables and C a countable set of constants.A rule is of the form head ← body, where head is an atom over R ∪ X ∪ C and body is a conjunction of positive or negative atoms over R ∪ X ∪ C.An example of such rule can be: When replacing all variables in a rule with concrete entities in KG, we get a дroundinд of the rule.For example, one grounding of Rule (2) can be: (Bob, hasParent, Ann), (Ann, дender, Female) A grounding with all triples existing in knowledge graph is a support of this rule.For examples, if (Bob, hasMother , Ann) ∈ K, (Bob, hasParent, Ann) ∈ K and (Ann, дender, Female) ∈ K, then grounding (3) is a support for rule (2).The results of rule learning is of the following form: α head ← body in which α ∈ [0, 1] is a confidence score assigned to the rule by the learning method.
Incorporating logical rules into other learning system such as embedding learning is called rule injection.One way for rule injection is adding regularizer or other constraints to entity and relation representations by propositionalizing the rules.Another way is adding constraints to the constituent relations' representation in rules without direct effect on entity representations.As we want to get new information of sparse entities through rules, we chose propositionalization in this paper for rule injection.

OWL 2 Web Ontology Language Axioms
In this paper, instead of learning general Horn rules or closed-path rules as previous works [9][48], we are more interested in ontology axioms, the main components of knowledge graph ontologies, because they are important for enriching semantics in KGs.
OWL2 Web Ontology Language, informally OWL22 , is an ontology language for Semantic Web with formally defined meaning and is a W3C recommendation.It defines multiple types of axioms, from which we select some of them as a guidance of rule structures.The selection is based on following principles: (i) the axioms are related with binary relations, the main components of rules in KG and (ii) they can infer new triples because rules are used to help add new information about sparse entities in this paper.Thus, we focus on object property expression axioms in OWL2 which are composed of binary relations in the context of KG.Finally, 7 types of object property expression axioms out of 14 are selected.The unselected axioms are mainly applied to help check the consistency in knowledge graph.
In OWL2, each axiom has its own condition revealing its semantics.The axiom is satisfied if its condition hold.We introduce the selected 7 types of object property expression axioms and their conditions in Table 1.We also translate the conditions of axioms into rule-form including a head and a body as introduced in the previous subsection.The translated rule forms are used to guide the structures of rules to be learned in this paper.

METHOD
Given a knowledge graph K = {E, R, T }, our goal is to learn embeddings and rules at the same time and make their advantages complement each other's difficulties.As discussed in Section 1, embedding learning methods suffer from the problem of data sparsity and rule learning methods have a very large search space.
In this paper, we propose a general framework IterE which learns embeddings and rules in an iterative manner and can be applied to many KGEs that are based on linear map assumption.It includes three main parts:(i) embedding learning, (ii) axiom induction and (iii) axiom injection.Figure 2 shows the key idea of IterE with these three iterative parts.
• Embedding learning learns entity embeddings E and relation embeddings R with a loss function L embeddinд to be minimized, calculated with input triples (s, r, o), each with a label related Table 2: Seven types of object property expression axioms selected from OWL2 ontology language.OP is the short for ObjectProperty.K denotes a KG and x, y, z are entity variables.v and M denote entity and relation embeddings respectively.I is identity matrix.with its truth value.The inputs are of two types: triples existing in K and triples that do not exist in K but are inferred by axioms.

Object Property Axioms Rule Form According to Linear Map Assumption Rule Conclusion
• Axiom Induction inducts a set of axioms A based on relation embeddings R from the embedding learning step, and assigns each axiom with a score s ax iom .
• Axiom Injection injects new triples about sparse entities in K to help improve their poor embeddings caused by insufficient training.The new triples are inferred from groundings of quality axioms with high scores in A from axiom induction.After axiom injection, with K updated, the process goes back to embedding learning again.
IterE is proposed based on the observation that embeddings learned with linear map assumption can fully support the axiom selected in this paper, while other assumptions such as translationbased assumption can't as pointed out in [50].Inherently, for each type of axioms, a meaningful conclusion can be drawn with relation embeddings according to linear map assumption.For example, considering axiom inverse(hasParent, hasChild), if (Mike, hasParent, John) exists in knowledge graph, according to the condition and rule form of inverse axiom in Table 1, another triple (John, hasChild, Mike) can be inferred.Suppose embeddings of Mike, John, hasParent and hasChild are v Mike , v J ohn , M hasChild and M has P ar ent respectively.According to the linear assumption for individual triples, we can get following two equations: v Mike M has P ar ent = v J ohn and v J ohn M hasChild = v Mike .With these two equations, another equation can be deduced: M has P ar ent M hasChild = I.Note that this conclusion equation is only related with two corresponding relation embeddings and is unrelated with concrete entities, thus it can be regarded as a general conclusion for axiom inverse(hasParent, hasChild).For other types of axioms, a general conclusion can be drawn in the same way.We list details of the conclusion for each type of axioms in Table 2.These conclusions about relation embeddings help guide axiom induction in this paper.

Embedding Learning
The input I of embedding learning is a set of triples with labels.Unlike previous KGEs, which only learn embeddings but not rules, With input I, the loss function of embedding learning is calculated by mean of cross entropy loss among all n input triples and the training goal is to minimize the following loss function: As we adopt linear map assumption, the score function ϕ(s, r , o) for each triple (s, r , o) is defined as: in which v s ∈ R 1×d , v o ∈ R 1×d are vector embeddings for subject and object entity.M r ∈ R d ×d is matrix embedding for relation and σ denotes the sigmoid function.d is the embedding dimension.The similarity between two vectors is evaluated via dot product.Our approach can be combined with many KGEs based on linear map assumption, such as DistMult [47] and ANALOGY [23].In this paper, we adopt ANALOGY as it achieves state-of-the-art results on link prediction.ANALOGY is proposed to deal with an important kind of reasoning, analogy reasoning, in embedding learning.It imposes analogical structures among embeddings and requires linear maps associated with relations to form a commuting family of normal matrices.Specifically, the relation matrix embeddings are constrained as block-diagonal matrices with each diagonal block is either a real scalar, or a 2-dimensional real matrix in the form of a −b b a , where both a and b are real scalars.
After embedding learning, a collection of entity embeddings E and relation embeddings R are learned and R will be used in axiom induction.

Axiom Induction
Given relation embedding R, axiom induction aims to induce a set of axioms A and assign a confidence score to each axiom in A.
To achieve this, IterE firstly generates a pool of possible axioms P with an effective pruning strategy.Then it predicts a score of each axiom a ∈ P based on calculation with R.  2 should be replaced with concrete relations.Axiom pool generation generates a pool of possible axioms P by searching for possible axioms with the number of support greater than 1.
One intuitive way of searching for possible axioms is generating axioms by replacing all relation variable in each type of axioms with each relation and then check the number of support for them, but this suffers from a huge search space.Another way is to generate axioms via random walk on knowledge graph while this method can't ensure coverage of axioms.Therefore axiom pool generation is not an easy task because it has to achieve a good balance between search space and coverage of highly possible axioms.
In this paper, we propose a pruning strategy combining traversing and random selection.There are two steps for generating possible axioms for each relation r ∈ R in IterE.
• step 2: complete partial axioms via randomly selecting k triples (e ′ , r , e ′′ ) ∈ T related with r .Replacing r ′ or r ′′ in partial axioms with relations that directly link to e ′ or e ′′ .
• step 3: search for support of each axiom, and add those axioms with number of support larger than 1 into axiom pool P.
The key point of whole process is choosing k, which is not trivial because a large k will lead to dramatic increase of search space while a small k will decrease coverage of axioms immediately.Thus a good choice of k should achieve followings: 1) the probability for P covering all highly possible axioms higher than t, named including probability.2) k is as small as possible.We defined the highly possible axioms as axioms with existing probability larger than p in this paper and we name p as minimum axiom probability.
We estimate the probability p(r , a x ) that r replaces the relation variable in head of axiom a x as: where N is the number of triples (e ′ , r , e ′′ ) ∈ T and n is the number of support for axiom a x in K. Thus choosing k triples to satisfy two requirements mentioned above can be formulated as follows: From above inequality in (8), following inequality can be reached: Thus with fixed p, t, the best choice of k should be the upper bound 10 15 ], this equation is monotone increasing and has small upper bound v u .For example, when p = 0.5, t = 0.95, v u is 6.
As the input of axiom pool generation is a fixed K during learning, axiom pool P only need to be generated once.3.2.2Axiom Predicting.Given current relation embedding R and axiom pool P, axiom predicting predicts a score s a for each axiom a ∈ P based on the rule conclusion for each type of axioms (column 4 in Table 2), in the form that M a 1 = M a 2 , where M a 1 and M a 2 are matrices either from a single matrix or a product of two matrices.As rule conclusions are derived from ideal linear map assumption, M a 1 and M a 1 usually are not equal but similar during training.Thus, instead of matching M a 1 and M a 2 , we estimate the truth of axiom a by similarity between M a 1 and M a 2 which is supposed to be related with Frobenius norm of their difference matrix: ) s a (F ) is then normalized as follows because the value of s a (F ) for different type of axioms vary dramatically: in which t is the type of axiom that a belongs to.s max (t) and s min (t) is the maximum and minimum Frobenius norm score among all type t axioms in P. s a ∈ [0, 1] is the final score for axiom a and the higher s a is the more confident that axiom a is.

Axiom Injection
Given knowledge graph K and possible axiom set A, axiom injection utilizes axiom' deductive capability to infer a set of new triples T ax iom for sparse entities and predict their labels.T ax iom will be injected into embedding learning to reduce the sparsity of entities.
3.3.1 Sparse entities.We evaluate the sparsity of entities as follows: sparsity(e) = 1 − f req(e) − f req min f req max − f req min (12) where f req(e) is the frequency of entity e participating in a triple, as subject or object entity.f req min and f req max are the minimum and maximum frequency among all entities.sparsity(e) ∈ [0, 1].sparsity(e) = 1 means e is the most sparse entity and sparsity(e) = 0 means the most frequent one.With sparsity(e) > θ spar sity , we regard entity e as a sparse entity, where θ spar sity is a sparse threshold.We use E spar se to denote the set sparse entity in K.
During axiom injection, triples related with sparse entities are injected into the input of embedding learning.In other words, in new inferred triples (s a , r a , o a ), either s a ∈ E spar se or o a ∈ E spar se or {s a , o a } ∈ E spar se .Thus after inferring all possible new triples, we filter those unrelated to sparse entities.3.3.2Predicting new triples and their labels.We utilize groundings to infer new triples, and the grounding for axioms considered in this paper can be generalized as the following form: (s a , r a , o a ) ← (s 1 , r 1 , o 1 ), (s 2 , r 2 , o 2 ), ..., (s n , r n , o n ) where the right side triples (s k , r k , o k ) ∈ T with k ∈ [1, n] are generated from the body of axiom rule form and (s a , r a , o a ) T is new inferred triples to be added into knowledge graph.
To predict label for (s a , r a , o a ), we first translate the grounding form in (13) in propositional logical expression: then we model groundings through t-norm based fuzzy logics [17].It regards the truth value of a propositional logical expression as a composition of constituent triples' truth value, through specific logical connectives (e.g.∧ and ⇒).For example, the truth value of a propositional logical expression (s 1 , r 1 , o 1 ) ⇒ (s 2 , r 2 , o 2 ) is determined by the true value of two component triples (s 1 , r 1 , o 1 ) and (s 2 , r 2 , o 2 ), via a composition defined by logical implication ⇒.
We follow the definition of composition associated with logical conjunction (∧), disconjunction (∨) and negation (¬) as [14]: in which a, b are logical expressions and π (x) is the truth value of x.Given these compositions, the truth value of any propositional logical expression can be calculated recursively through Equation ( 15)- (18).For example, applying to propositional logical expression of grounding д : (s 1 , r 1 , o 1 ) ∧ (s 2 , r 2 , o 2 ) ⇒ (s a , r a , o a ): To predict the truth value π (s a , r a , o a ) for triples (s a , r a , o a ), inferred by axiom a according to grounding д a , based on t-norm fuzzy logics, π (s a , r a , o a ) can be calculated according to π (д) and π (s 1 , r 1 , o 1 ), π (s 2 , r 2 , o 2 ), ..., π (s n , r n , o n ) ∈ д a .In former example, For the truth value of axiom a's grounding д, we evaluate it as the score of a generated by axiom induction, namely π (д a ) = s a .For the truth value of triples existing in knowledge graph, we evaluate them with their training labels, thus π (s, r, o) = 1 for (s, r , o) ∈ T , because existing triples are absolutely true.With these two types of truth value assignment, we can easily get the following result for triples (s a , r a , o a ) inferred by any type of axioms a: After axiom injection, a set of new triples T axiom = {(s a , r a , o a )| s a ∈ E spar se or o a ∈ E spar se } are inferred via quality axioms and each new triple (s a , r a , o a ) is labeled with l sr o = s a .Thus the input of embedding learning was updated, I = {((s, r , o), l sr o )|(s, r, o) ∈ T ∧ T axiom }.Then the process goes back to embedding learning.

EXPERIMENT
During experiments, we want to explore following questions: 1) whether axioms really help sparse entity embedding learning?To do this, we evaluate the quality of embeddings on link prediction task which is widely applied in previous knowledge graph embedding works; 2) whether embeddings really help rule learning overcome the huge search space and improve the quality of learned rules?To do this, we evaluate the efficiency of axiom learning with learning time and the quality with number and percentage of high quality axioms the method generate; 3) how does the iterative manner affect embedding learning and rule learning process?To this end, we show the changes of link prediction results, rule qualities and the number of triples injected along with different number of iterations.

Dataset
Four datasets are used in experiment, including WN18-sparse, WN18RRsparse, FB15k-sparse, and FB15k-237-sparse.They are generated from WN18 [3], WN18RR [7], FB15k [3] and FB15-237 [34], four datasets that are commonly used in previous knowledge graph embedding work.WN18 and WN18RR are subsets of WordNet, a lexical knowledge graph for English.FB15k and FB15k-237 are subsets of a large collaborative knowledge base Freebase.The statistics of datasets are listed in Table 3.
We generate the sparse version of these datasets via changing the valid and test datasets which will be used for link prediction evaluation.In link prediction experiments, we want to explore whether axioms really contribute to sparse entity embeddings.Therefore, we only keep triples in test and valid dataset which involve at least one sparsity entity.In other words, for a test or valid triple (s, r , o), if either s or o is or both are sparse entity (Equation ( 12)), the triple will be kept in sparse dataset, otherwise, it will be filtered.When deciding sparse entity hyperparameter θ , intuitively a principle is considered: the percentage of left triples in valid and test set should not be larger than 80% or less than 20% of original valid and test set for all datasets.Thus we choose θ = 0.995 and regard entities e with sparsity(e) > 0.995 as sparse entity in this paper.Table 3 also shows the percent of triples left.

Training Details
For embedding learning, the number of negative samples is set to 6 and the number of scalars on the diagonal of each relation matrix is set to d 2 where d is embedding dimension.We initialize embeddings with uniform distribution U(−0.1, 0.1).
For axiom induction, we set the minimum axiom probability p = 0.5 and the including probability t = 0.95 for axiom pool generation.Based on these settings, the number of related triples selected for each relation is set as k = 6 according to Equation ( 9).Details of axiom pools generated for all datasets are shown in Table 4, where we can see that the number of possible axioms in FB15k and FB15k-237 are much larger than WN18 and WN18RR because the number of axioms is highly related to the diversity of relations.For axiom injection, considering that axioms with high scores are more reliable and less possible to introduce noise, we set a threshold θ for each dataset and regard axioms with scores s ax iom > θ as high quality axioms.We also set a maximum inferred triples m for axioms in each dataset, if one axiom will infer a lot of triples we ignore it because a huge number of triples inferred by one axiom will change the data distribution significantly and make the embedding training unstable.
During one iteration learning, we first train embedding learning for 10 epochs, and then conduct axiom induction and injection once.The maximum training iteration is set to 10 for WN18RR-sparse and FB15k-237-sparse and 50 for WN18-sparse and FB15k-sparse.We use Adam algorithm [19] during optimization with learning rate lr = 0.001.We apply grid search for the best hyperparameters based on the filter MRR on the validation dataset, with combinations from embedding dimension d ∈ {100, 200} and l 1 regularizer weigh λ ∈ {10 −4

Embedding Evaluation
We evaluate the quality of embeddings on link prediction tasks.Link prediction aims to predict the missing entity when given the other entity and relation in a triple, including subject entity prediction (?, r , o) and object entity prediction (s, r , ?). (?, r , o) with right answer s, we first fit subject entity position with each entity e ∈ E and thus get a set of triples T sub jectpr edict ion = {(e, r , o)|e ∈ E}.Then we calculate the score for each triple in T sub jectpr edict ion according to Equation ( 6) and rank their scores in descending order.Thus the entity e in (e, r , o) with a higher rank is a more possible prediction.To evaluate how good the prediction is, we use the rank of (s, r, o) among all triples in T sub jectpr edict ion as subject entity prediction evaluation result for (s, r , o), namely subject entity prediction rank rank s (?, r , o).The object entity prediction task is done in the same way and will get objection entity rank rank o (s, r , ?).The final prediction rank rank(s, r , o) for (s, r , o) is the average of subject and object prediction rank:

Evaluation metrics. Given subject entity prediction task
Aggregating prediction rank for all test triples, we applied mean reciprocal predicting rank (MRR) and the percentage of predicting ranks within n (Hit@n) to evaluate the whole prediction result.These two metrics are widely used in previous work [35][23].Generally, a higher MRR or Hit@n indicates a better prediction result.We also apply f ilter and raw setting.As mentioned in [41], when fitting subject or object entity position with other entities, we may generate triples existing in knowledge graph which are known to be true.It is not wrong if these triples are ranked higher than current test triple.With f ilter setting, we filter the generated triples that exist in both train/valid/test dataset before ranking but not current test triple.raw means the setting of NO filtering.

Baselines.
For baselines, one method from translation-based assumption TransE 3 and three methods based on linear map assumption, DistMult, ComplEx, and ANALOGY 4 are considered.

Results and analysis.
To show how axiom helps sparse entity embeddings, we adopt two strategies: (1) Firstly, we evaluate how axioms directly improve entity embedding quality and compare link prediction results from our method, denoted as IterE in Table 5, with other baselines directly.(2) Secondly, we evaluate how axioms can help improve sparse entity link prediction utilizing its deductive ability.Thus we compare prediction results with embeddings and axioms, denoted as IterE+axioms in Table 5 with IterE and other methods.In IterE+axioms, if the test triple (s, r, o) are inferred by axioms during axiom injection, which means (s, r, o) ∈ T axiom , we regard it as correct and mark its prediction rank as 1.
The link prediction results are shown in Table 5.We analyze the results as follows: Firstly, the link prediction results of IterE competitive to ANALOGY, which means most of the triples injected into embedding learning are not noise, indicating that learning axioms from embeddings works well.Secondly, IterE outperforms baselines on WN18RR-sparse and FB15k-237-sparse, while are slightly improved on WN18-sparse and FB15k-sparse.These indicate that IterE helps 3 the code for TransE is from https://github.com/thunlp/OpenKE.We train all dataset with learning rate 0.01, margin 1.0 and dimension 100 for maximum 3500 iterations. 4The code for DistMult, ComplEx and ANALOGY is from https://github.com/quark0/ANALOGY.We train them with the same parameter setting as [23] for maximum 500 iterations.Table 5: Link prediction results with MRR and Hit@n on WN18RR-sparse and FB15k-237-sparse.Underlined scores are the better ones between ANALOGY and IterE(ANALOGY).Boldface scores are the best results among all methods.

WN18-sparse
FB15k-sparse MRR MRR Hit@1 Hit@3 Hit10 MRR MRR Hit@1 Hit@3 Hit10 (filter) (raw) (filter) (filter) (filter) (filter) (raw) (filter) (filter) (filter) TransE [3] 41 sparse entities much more in WN18RR-sparse and FB15k-237-sparse than in WN18RR-sparse and FB15k-sparse.This is quite reasonable in our opinion, because among these four datasets, WN18RR-sparse and FB15k-237-sparse are more challenging and more sparse.They are created from WN18-sparse and FB15k-sparse via removing one relation of all inverse relation pair in the dataset and also their related triples because inverse relation pair is a significant pattern in these two datasets as first noted in [34].Thirdly, the results of IterE+axioms are improved compared with IterE in all datasets, especially on the most complex and sparse dataset FB15k-237-sparse.It indicates that the deductive capability of axioms can help the prediction of sparse entities further.
From the evaluation of link prediction, we can conclude that: (1) by injecting new triples for sparse entities, axioms help improve the quality of sparse entity embeddings and are more helpful in sparse KGs.(2) Combining axioms and embeddings together to predict missing links works better than using embeddings only.Both the deductive capability of axioms and the inductive capability of embeddings contribute to prediction and complement each other.

Rule Evaluation
We evaluate the learned rules/axioms from two perspectives: efficiency and quality.We compare our method with AMIE+ [9]  5 which is an improved rule mining system of widely used AMIE [10].4.4.1 Evaluation metrics.The efficiency of rule learning is evaluated by learning time.The quality of rule learning is evaluated with the number of high quality rule (HQr) and their percentage.The quality of rules are evaluated by head coverage(HC) which is commonly used in pervious work, such as [10] and [29].Head 5 we run AMIE+ code from https://www.mpi-inf.mpg.de/departments/databases-andinformation-systems/research/yago-naga/amie/coverage for rule rul is defined as follows: #(e, e ′ ) : support(rul) ∧ head(rul)(e, e ′ ) #(e, e ′ ) : head(rul)(e, e ′ ) in which head(rul)(e, e ′ ) = {(e, r, e ′ ) ∈ T } if the head atom of rul is (X, r , Y).And support(rul) is the supports for rul.We regard high quality rules as the rules with HC > 0.7 during test.4.4.2Results and analysis.Rule evaluation results are shown in Table 6.According to the time used to generate rules among all datasets, we can see that IterE learns rules more efficiently.For example, with FB15k-sparse and FB15k-237-sparse datasets, IterE costs 10 times less than AMIE+.This shows our pruning strategy works well.Among 4.72 seconds IterE cost for FB15k-237-sparse dataset, there is 4.55 seconds used for axiom pool generation and 0.17 seconds for axiom score calculation, namely axiom score calculation only cost 3.6% of the time.This means calculating axiom scores via embeddings is super efficient.We didn't include the time of embedding learning during this evaluation, as embedding learning is not devised mainly for rule learning, but for link prediction.
The number of high quality rules shows that IterE generates more high quality rules than AMIE+ for each dataset and also achieves a higher percentage.This indicates that our axiom pool generation can filter meaningless axioms(rules) and achieve a good balance between small search space and coverage of highly possible axioms.Further more, Figure 4 shows the changes of high quality axioms coverage and axiom percentage with different axiom score thresholds from IterE.For example, in 4(c), with axiom threshold 0.9, which means we select axioms with s a > 0.9 from IterE for axiom injection, there are 23.4% axioms selected among all axioms and 46.4% high quality axioms included.And in 4(d), with axiom threshold 0.9, there are 53.9%axioms selected and 76.5% high quality axioms included.It illustrates that axiom scores calculated from embeddings are reliable because it is consistent with rule evaluation results.The results in 4(a) and 4(b) for WN18-sparse and WN18RRsparse are not obvious because a few relations are contained in these two datasets and the number of learned rules is limited.
From rule evaluation results, we can conclude that (1) embeddings together with axiom pool generation help rule learning overcome large search space problem and improve rule learning efficiency, and (2) they also improve rule learning qualities and rules' reliable scores generated based on calculation with embeddings.

Iterative learning
To explore how iterative training improves embedding and rule learning during training, we show link prediction results on FB15k-237-sparse and number of injected triples at different iterations in Figure 5.

Case study
In Table 7, we give a case study with 3 test triple examples which are inferred by axioms during training.Using the third one as an example, the test triple is (Jenny_McCarthy, /type_o f _union, Marriage).Jenny_McCarthy is a person with only a few triples in KG and is a sparse entity with sparsity 0.997.Marriage is a type of union for people which has many links with individual person and its sparsity is 0.607.The subject prediction rank (?, /type_of_union, Marriage) in ANALOGY is 4199 and the object prediction (Jenny_McCarthy, /type_o f _union, ?) is 2, from which we can see that the prediction for a sparse entity is much worse than a frequent entity.In IterE, this triple is predicted by rule (X, /type_o f _union, Z) ← (X, spouse, Y), (Y, /type_o f _union, Z) because there are (Jenny_McCarthy, spouse, Jim_Carrey) and (Jim_Carrey, /type_o f _union, Marriage) in KG which can compose a grounding for previous rules.Thus during training, the test triple is inferred by IterE and injected into embedding learning.Thus the subject prediction is improved from 4199 to 33 compared with ANALOGY and the object prediction keeps the same.When we predict with both embeddings and axioms (IterE + axioms), the subject prediction is improved from 33 to 1 and the object prediction from 2 to 1 compared with IterE.Other two cases in Table 7 perform similar to the third one.
These examples show that: (1) adding triples that can be inferred by axioms back into training improves the prediction results of related sparse entities without hurting the results for non-sparse entities.(2) the deductive ability of axioms helps ensure truth value of triples with sparse entities, which, in our opinion, overcomes uncertainties and noises that effect embedding prediction.

Figure 1 :
Figure 1: The numbers and mean reciprocal rank of different frequency entities based on ANALOGY results on FB15k-237.
positive input triple in IterE contains two parts: triples (s, r, o) ∈ T existing in original knowledge graph and triples (s, r, o) ∈ T axiom , where T axiom is a set of triples inferred by axioms learned from embeddings.Negative input triples (s ′ , r ′ , o ′ ) ∈ T neдat ive are generated by randomly replacing s or o with e ∈ E or replacing r with r ′ ∈ R for (s, r , o) ∈ T .Thus the input set I in IterE is as follows: I = {((s, r, o), l sr o )|((s, r , o)) ∈ T ∧ T axiom ∧ T neдat ive } (4) where l sr o is the label for triple (s, r , o) to evaluate its truth value.l sr o = 1 for (s, r, o) ∈ T and l sr o = 0 for (s, r , o) ∈ T neдat ive .For triples (s, r , o) ∈ T axiom we assign l sr o = π (s, r, o), where π (s, r , o) is triple truth value predicted by axiom injection (Section 3.3).

Figure 3 :
Figure 3: Detailed example for the process of IterE.3.2.1 Axiom pool generation.Before calculating axiom scores with relation embedding R, relation variables r, r 1 , r 2 , and r 3 in Table2should be replaced with concrete relations.Axiom pool generation generates a pool of possible axioms P by searching for possible axioms with the number of support greater than 1.One intuitive way of searching for possible axioms is generating axioms by replacing all relation variable in each type of axioms with each relation and then check the number of support for them, but this suffers from a huge search space.Another way is to generate axioms via random walk on knowledge graph while this method can't ensure coverage of axioms.Therefore axiom pool generation is not an easy task because it has to achieve a good balance between search space and coverage of highly possible axioms.In this paper, we propose a pruning strategy combining traversing and random selection.There are two steps for generating possible axioms for each relation r ∈ R in IterE.•step 1: generate axioms or partial axioms: ReflexiveOP(r ), SymmetricOP(r ), TransitiveOP(r ), EquivalentOP(r ′ , r ), subOP(r ′ , r ), inverseOP(r ′ , r ) and subOP(OPChain(r ′ , r ′′ ), r ), , in which {r ′ , r ′′ } are relation variables to be replaced with relations in KG and r participants in the head of their rule forms.•step 2: complete partial axioms via randomly selecting k triples (e ′ , r , e ′′ ) ∈ T related with r .Replacing r ′ or r ′′ in partial axioms with relations that directly link to e ′ or e ′′ .•step 3: search for support of each axiom, and add those axioms with number of support larger than 1 into axiom pool P.The key point of whole process is choosing k, which is not trivial because a large k will lead to dramatic increase of search space while a small k will decrease coverage of axioms immediately.Thus a good choice of k should achieve followings: 1) the probability for P covering all highly possible axioms higher than t, named including probability.2) k is as small as possible.We defined the highly possible axioms as axioms with existing probability larger than p in this paper and we name p as minimum axiom probability.

Figure 5 :
Figure 5: 5(a) shows link prediction results, MRR and Hit@10, in different iterations.5(b) shows the number of triples injected into embedding learning in different iterations.

Figure 5
Figure5shows that the prediction results including Hit@10 and MRR become better as the training iteration increases and generally the number of injected triples increases during training and

Table 1 :
Conditions for object property expression axioms in OWL2 and translated rule formulation.OP refers to "ObjectProperty".OPE, with or without subscript, denotes Object Property Expression and x, y, z are entity variables.△ I is a nonempty set called object domain.• OP is an object property interpretation function.When translating axioms into rule forms according to condition, we replace OPE in axioms with binary relation r in the context of knowledge graph and the number of OPE in EquivalentOP and the OPChain in SubOP is set to 2.

Table 3 :
Statistics of datasets.

Table 6 :
Rule evaluation results.