The test set derived from TutorialBank for evaluating improved concept embeddings for learning prerequisite chains consists of:
- A list of 208 topics extracted from files (expanded to 227 topics, including common acronyms and alternate formats), which act as nodes for co-occurrence graphs and prerequisite edges. An ID is assigned to each topic.
- A list of annotated prerequisite relations between the topics using the format $$id1, id2, prereq$$, where $$prereq$$ is 1 if $$id1$$ is a prerequisite of $$id2$$ and 0 otherwise. Out of 43,056 possible prerequisite pairs, 913 pairs have $$prereq$$ equal to 1.

Improved Concept Embeddings for Learning Prerequisite Chains: Test set

A corpus of total 7,472 text files, consisting of two parts:
- LectureBank(Li et al., 2018): a manually-collected dataset of 1352 lecture slide presentations from 60 courses covering 5 domains: Natural Language Processing (nlp), Machine Learning (ml), Artificial Intelligence(ai), Deep Learning (dl), and Information Retrieval (ir). 
- TutorialBank (Fabbri et al., 2018): a manually-collected dataset of over 6000 resources, ranging from HTML pages (.txt) to lecture slides and textbooks (.pdf, .pptx, and .ppt), mainly in the domain of NLP.

Improved Concept Embeddings for Learning Prerequisite Chains: Training set

University of Michigan - Ann Arbor

- Poincaré embeddings (Nickel and Kiela, 2017) is a method that better fit for data with latent hierarchies than other embedding models using the Euclidean vector space.
- Dhingra et al. (2018) introduced a method to obtain Poincaré word embeddings from natural language text.
- This paper combines the two methods above to obtain prerequisite relations between concepts from two datasets in the domain of Natural Language Processing. 

Improved Concept Embeddings for Learning Prerequisite Chains: Methods

Löffel, E. P. (2019). Improved Concept Embeddings for Learning Prerequisite Chains

Paper: https://pdfs.semanticscholar.org/6073/0900a80fcee29e9059fd293a1fa38742abf3.pdf

Github: https://github.com/eloffel/improved_embeddings

Improved Concept Embeddings for Learning Prerequisite Chains: Reference

- The embedding models using the Euclidean vector space are unable to capture hierarchical relationships, which is not optimized for embedding structures like graphs and trees. Nickel and Kiela (2017) proposed a model that embeds concepts into a hyperbolic space, which can be thought of as a continuous version of trees structure and therefore provides a better fit to embed data with latent hierarchies.
- Poincaré ball model represents the entire hyperbolic space within the border of an n-dimensional sphere. This model, referred to as Poincaré Embeddings, generates the embeddings having as its input a weighted directed graph $G = \{(b, a)\}$, in which each node represents a concept and a directed edge between a and b means that a is a hypernym of b, in other words, b is a prerequisite concept of a. 

Improved Concept Embeddings for Learning Prerequisite Chains: Poincaré embeddings

- Learning rate = 0.2
- Number of negative examples = 80 
- Number of epochs = 100
- Epochs of burn-in = 10 
- Number of dimensions = 2, 3, 5, 10, 50, 100

Improved Concept Embeddings for Learning Prerequisite Chains: Model parameters

Improved Concept Embeddings for Learning Prerequisite Chains: Evaluation methods

To generate Poincaré embeddings from a natural language text corpus, the author converted the text to a list of directed edges in the format $$u, v, \text{weight}$$, where $$v$$ is a hypernym of $$u$$, as input to the Poincaré embeddings algorithm, using the method described by Dhingra et al. (2018). They constructed a co-occurrence graph $$G = \{(u, v)\}$$ that includes all pairs of words that occur within a fixed window of each other. Each edge $$(u, v)$$ in the graph has a weight $$w = f^c$$, where $$f$$ is the frequency of the co-occurrence pairs in the corpus and $$c$$ is a downsampling constant.

Learn Before

Related

Learn After