Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones
Here we present ConE (Cone Embedding for knowledge graphs), the first knowledge graph (KG) embedding method that can capture the transitive closure properties of heterogeneous hierarchical relations as well as other non-hierarchical properties.
Motivation
Hierarchical relations are prevalent and indispensable for organizing human knowledge captured by a knowledge graph (KG). The key property of hierarchical relations is that they induce a partial ordering over the entities, which needs to be modeled in order to allow for hierarchical reasoning. However, existing geometry embedding works (Poincare, Hyperbolic cone, Box) that consider hierarchical relations do not take into account existing non-hierarchical relations, and can only be applied to graphs with a single hierarchical relation type. Motivated by the transitivity of nested angular cones, we propose ConE model that embeds entities into hyperbolic cones and models relations as transformations between the cones. In particular, ConE uses cone containment constraints in different subspaces of the hyperbolic embedding space to capture multiple heterogeneous hierarchies.
Figure 1: Multiple heterogeneous hierarchies in knowledge graph. There are three categories of relations:
non-hierarchical relation (sister term), hypernym (partOf) and hyponym relation (subClass). Relations induce multiple independent hierarchies.
Method
ConE embedding space is the product space of $d$ hyperbolic planes and ConE learns a different transformation in each hyperbolic plane. ConE uses restricted rotation in an assigned relation-specific subspace to model each hierarchical relation $r$ and enforces cone containment constraint in the subspace so that partial ordering of cones is preserved in the subspace. For hyperbolic planes not in the subspace, we use a general rotation to model $r$.
Figure 2: ConE model overview.
Rotation. we adopt rotation transformation to model non-hierarchical properties.
Figure 3: Transformations adopted by ConE model.
Restricted rotation. We use restricted rotation transformation to model hierarchical relations. Restricted rotation contains two steps: 1. Scaling from parent cone to child cone; 2. Rotation (maximum angle given by half aperture of parent cone), ensuring that the child cone still resides in the parent cone.
Emperical Results
Given a KG containing many hierarchical and non-hierarchical relations, our experiments evaluate: (A) Performance of ConE on hierarchical reasoning task of predicting if entity $h_1$ is an ancestor of entity $h_2$, namely ancestor-descendant prediction. (B) Performance of ConE on generic KG completion tasks, namely knowledge graph completion. (C) Performance of ConE on hierarchical reasoning task of identifying the lowest common ancestor (LCA) of two entities, namely LCA prediction.
We conduct ancestor-descendant prediction on three hierarchcial datasets.
We further apply ConE to knowledge graph completion task, across four KG benchmarks.
Given two entities, we want to find the most distinguishable feature they have in common, e.g., LCA(WinePalm, SugarPalm)=PalmTree. LCA prediction is conducted on WN18RR dataset.
Please refer to our paper for detailed explanations and more results.
Code and Datasets
The code and datasets are included in the code repository on Github.Contributors
The following people contributed to this work:Yushi Bai, Tsinghua University
Rex Ying, Stanford University
Hongyu Ren, Stanford University
Jure Leskovec, Stanford University