Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones

Here we present ConE (Cone Embedding for knowledge graphs), the first knowledge graph (KG) embedding method that can capture the transitive closure properties of heterogeneous hierarchical relations as well as other non-hierarchical properties.

Motivation

Hierarchical relations are prevalent and indispensable for organizing human knowledge captured by a knowledge graph (KG). The key property of hierarchical relations is that they induce a partial ordering over the entities, which needs to be modeled in order to allow for hierarchical reasoning. However, existing geometry embedding works (Poincare, Hyperbolic cone, Box) that consider hierarchical relations do not take into account existing non-hierarchical relations, and can only be applied to graphs with a single hierarchical relation type. Motivated by the transitivity of nested angular cones, we propose ConE model that embeds entities into hyperbolic cones and models relations as transformations between the cones. In particular, ConE uses cone containment constraints in different subspaces of the hyperbolic embedding space to capture multiple heterogeneous hierarchies.




Figure 1: Multiple heterogeneous hierarchies in knowledge graph. There are three categories of relations:
non-hierarchical relation (sister term), hypernym (partOf) and hyponym relation (subClass). Relations induce multiple independent hierarchies.

Method

ConE embedding space is the product space of $d$ hyperbolic planes and ConE learns a different transformation in each hyperbolic plane. ConE uses restricted rotation in an assigned relation-specific subspace to model each hierarchical relation $r$ and enforces cone containment constraint in the subspace so that partial ordering of cones is preserved in the subspace. For hyperbolic planes not in the subspace, we use a general rotation to model $r$.



Figure 2: ConE model overview.

Here we introduce a combination of two transformations for modeling knowledge graph triples in our design:

Emperical Results

Given a KG containing many hierarchical and non-hierarchical relations, our experiments evaluate: (A) Performance of ConE on hierarchical reasoning task of predicting if entity $h_1$ is an ancestor of entity $h_2$, namely ancestor-descendant prediction. (B) Performance of ConE on generic KG completion tasks, namely knowledge graph completion. (C) Performance of ConE on hierarchical reasoning task of identifying the lowest common ancestor (LCA) of two entities, namely LCA prediction.

Please refer to our paper for detailed explanations and more results.

Code and Datasets

The code and datasets are included in the code repository on Github.

Contributors

The following people contributed to this work:
Yushi Bai, Tsinghua University
Rex Ying, Stanford University
Hongyu Ren, Stanford University
Jure Leskovec, Stanford University

References

Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones. Y. Bai*, R. Ying*, H. Ren, J. Leskovec. Proc. 35th Annual Conference on Neural Information Processing Systems (NeurIPS 2021).