TutorialBank's resources: 
- A list of 208 topics extracted from the files, which will be used as the nodes for the co-occurrence graphs and for the prerequisite edges. The list of 208 topics was expanded to 227 topics, including common acronyms and different writing formats. Then, an id was assigned to each topic. 
- A list of annotated prerequisites relations between topics above with the format $id1,id2,prereq$, where prereq is 1 if $id1$ is a prerequisite of $id2$ and 0 otherwise. Out of all the 43,056 possible prerequisite pairs, there are only 913 pairs with prereq equal to 1.

Improved Concept Embeddings for Learning Prerequisite Chains: Test set

A corpus of total 7,472 text files, consisting of two parts:
- LectureBank(Li et al., 2018): a manually-collected dataset of 1352 lecture slide presentations from 60 courses covering 5 domains: Natural Language Processing (nlp), Machine Learning (ml), Artificial Intelligence(ai), Deep Learning (dl), and Information Retrieval (ir). 
- TutorialBank (Fabbri et al., 2018): a manually-collected dataset of over 6000 resources, ranging from HTML pages (.txt) to lecture slides and textbooks (.pdf, .pptx, and .ppt), mainly in the domain of NLP.

Improved Concept Embeddings for Learning Prerequisite Chains: Training set

University of Michigan - Ann Arbor

- Poincaré embeddings (Nickel and Kiela, 2017) is a method that better fit for data with latent hierarchies than other embedding models using the Euclidean vector space.
- Dhingra et al. (2018) introduced a method to obtain Poincaré word embeddings from natural language text.
- This paper combines the two methods above to obtain prerequisite relations between concepts from two datasets in the domain of Natural Language Processing. 

Improved Concept Embeddings for Learning Prerequisite Chains: Methods

Löffel, E. P. (2019). Improved Concept Embeddings for Learning Prerequisite Chains

Paper: https://pdfs.semanticscholar.org/6073/0900a80fcee29e9059fd293a1fa38742abf3.pdf

Github: https://github.com/eloffel/improved_embeddings

Improved Concept Embeddings for Learning Prerequisite Chains: Reference

Improved Concept Embeddings for Learning Prerequisite Chains: Datasets

- The embedding models using the Euclidean vector space are unable to capture hierarchical relationships, which is not optimized for embedding structures like graphs and trees. Nickel and Kiela (2017) proposed a model that embeds concepts into a hyperbolic space, which can be thought of as a continuous version of trees structure and therefore provides a better fit to embed data with latent hierarchies.
- Poincaré ball model represents the entire hyperbolic space within the border of an n-dimensional sphere. This model, referred to as Poincaré Embeddings, generates the embeddings having as its input a weighted directed graph $G = \{(b, a)\}$, in which each node represents a concept and a directed edge between a and b means that a is a hypernym of b, in other words, b is a prerequisite concept of a. 

Improved Concept Embeddings for Learning Prerequisite Chains: Poincaré embeddings

To generate Poincaré embeddings from a natural language text corpus, the author converted the text to a list of directed edges in the format $u,v,weight$, where $v$ is a hypernym of $u$, as input to the Poincaré embeddings algorithm, using the method described by Dhingra et al. (2018).  They constructed a co-occurrence graph $G = \{(u, v)\}$ that includes all pairs of words that occur within a fixed window of each other. Each edge $(u, v)$ in the graph has a weight $w = f^c$, where $f$ is the frequency of the co-occurrence pairs in the corpus and $c$ is a downsampling constant. 

Improved Concept Embeddings for Learning Prerequisite Chains: Generating Poincaré embeddings from natural-language text corpora

- Learning rate = 0.2
- Number of negative examples = 80 
- Number of epochs = 100
- Epochs of burn-in = 10 
- Number of dimensions = 2, 3, 5, 10, 50, 100

Learn Before

Related

Learn After