The loss function used in this article is $$\mathcal{L}(\Theta, d)$$, where embeddings are represented as $$\Theta = \{\theta_i\}_{i=1}^n$$ and $$d$$ is the Poincaré distance. This loss function minimizes the Poincaré distance between similar concepts while maximizing the distance between different concepts.

University of Michigan - Ann Arbor

Google

- The embedding models using the Euclidean vector space are unable to capture hierarchical relationships, which is not optimized for embedding structures like graphs and trees. Nickel and Kiela (2017) proposed a model that embeds concepts into a hyperbolic space, which can be thought of as a continuous version of trees structure and therefore provides a better fit to embed data with latent hierarchies.
- Poincaré ball model represents the entire hyperbolic space within the border of an n-dimensional sphere. This model, referred to as Poincaré Embeddings, generates the embeddings having as its input a weighted directed graph $G = \{(b, a)\}$, in which each node represents a concept and a directed edge between a and b means that a is a hypernym of b, in other words, b is a prerequisite concept of a. 

Improved Concept Embeddings for Learning Prerequisite Chains: Poincaré embeddings

The distance between two vectors $$u$$ and $$v$$ in the Poincaré sphere is $$d(u, v) = \cosh^{-1}\left(1+2\frac{||u-v||}{(1-||u||^2)(1-||v||^2)}\right)$$, where all vectors are within a distance of 1 from the origin.

Improved Concept Embeddings for Learning Prerequisite Chains: Poincaré Space Distance

Nickel, M., & Kiela, D. (2017). Poincaré embeddings for learning hierarchical representations. In Advances in neural information processing systems (pp. 6338-6347).

References for Poincaré embeddings

Löffel, E. P. (2019). Improved Concept Embeddings for Learning Prerequisite Chains

Paper: https://pdfs.semanticscholar.org/6073/0900a80fcee29e9059fd293a1fa38742abf3.pdf

Github: https://github.com/eloffel/improved_embeddings

Learn Before

Related