Here we present **ConE** (Cone Embedding for knowledge graphs), the first knowledge graph (KG) embedding method that can capture the **transitive closure properties of heterogeneous hierarchical relations** as well as other **non-hierarchical properties**.

Hierarchical relations are prevalent and indispensable for organizing human knowledge captured by a knowledge graph (KG).
The key property of hierarchical relations is that they induce a partial ordering over the entities, which needs to be modeled in order to allow for hierarchical reasoning.
However, existing geometry embedding works (Poincare, Hyperbolic cone, Box) that consider hierarchical relations do not take into account existing non-hierarchical relations, and can only be applied to graphs with a single hierarchical relation type.
Motivated by the transitivity of nested angular cones, we propose **ConE** model that embeds entities into hyperbolic cones and models relations as transformations between the cones.
In particular, ConE uses cone containment constraints in different subspaces of the hyperbolic embedding space to capture multiple heterogeneous hierarchies.

**
Figure 1: Multiple heterogeneous hierarchies in knowledge graph.** There are three categories of relations:

non-hierarchical relation (*sister term*), hypernym (*partOf*) and hyponym relation (*subClass*). Relations induce multiple independent hierarchies.

ConE embedding space is the product space of $d$ hyperbolic planes and ConE learns a different transformation in each hyperbolic plane. ConE uses **restricted rotation** in an assigned **relation-specific subspace** to model each hierarchical relation $r$ and enforces cone containment constraint in the subspace so that partial ordering of cones is preserved in the subspace.
For hyperbolic planes not in the subspace, we use a general **rotation** to model $r$.

**Figure 2: ConE model overview.**

**Rotation.**we adopt rotation transformation to model non-hierarchical properties.

**Figure 3: Transformations adopted by ConE model.****Restricted rotation.**We use restricted rotation transformation to model hierarchical relations. Restricted rotation contains two steps: 1. Scaling from parent cone to child cone; 2. Rotation (maximum angle given by half aperture of parent cone), ensuring that the child cone still resides in the parent cone.

Given a KG containing many hierarchical and non-hierarchical relations, our experiments evaluate:
**(A)** Performance of ConE on hierarchical reasoning task of predicting if entity $h_1$ is an ancestor of entity $h_2$, namely ancestor-descendant prediction.
**(B)** Performance of ConE on generic KG completion tasks, namely knowledge graph completion.
**(C)** Performance of ConE on hierarchical reasoning task of identifying the lowest common ancestor (LCA) of two entities, namely LCA prediction.

We conduct ancestor-descendant prediction on three hierarchcial datasets.

We further apply ConE to knowledge graph completion task, across four KG benchmarks.

Given two entities, we want to find the most distinguishable feature they have in common, e.g.,

*LCA(WinePalm, SugarPalm)=PalmTree*. LCA prediction is conducted on WN18RR dataset.

Please refer to our paper for detailed explanations and more results.

The following people contributed to this work:

Yushi Bai, Tsinghua University

Rex Ying, Stanford University

Hongyu Ren, Stanford University

Jure Leskovec, Stanford University

Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones. Y. Bai*, R. Ying*, H. Ren, J. Leskovec. * Proc. 35th Annual Conference on Neural Information Processing Systems (NeurIPS 2021)*.