1. Synthetic
2. ASSIST2009
3. ASSIST2015
4. ASSISTChall
5. STATICS2011

Dataset (A Self-Attentive model for Knowledge Tracing)

**Metrics** - prediction task is binary classification task and the authors use AUC (area under curve) to measure the performance.
**Approaches** - their created model is compared to previous models, namely, DKT (deep knowledge tracing), DKT+, DKVMN(dynamic key value memory network).
**Model training and parameter selection** the dataset was divided into train/test sets in the following way: 80% goes to train and 20% to test. Different size of hidden state have been tried: {50, 100, 150, 200}. The model was implemented in tensorflow and adam optimizer with 0.001 was utilized.  

Evaluation Methodology (A Self-Attentive model for Knowledge Tracing)

San Diego State University

1. Introduction
2. Proposed method
3. Experimental setting
4. Results and Analysis
5. Conclusion and Future Work

A Self-Attentive model for Knowledge Tracing

Paper: https://arxiv.org/pdf/1907.06837.pdf

Code: https://github.com/shalini1194/SAKT/tree/master/2019-EDM

Reference for A Self-Attentive model for knowledge Tracing

Deep learning based knowledge tracing(DKT), namely, modeling student knowledge state based on recurrent neural networks,  have shown better results than the BKT (Bayesian Knowledge Tracing) and PFA (Performance Knowledge Tracing).  (**NOTE** without human engineered features)
 
Here we transform fixed length inputs using  one-hot encoding and then they are passed to the hidden layer. The hidden state can be thought as a latent knowledge state of the student. Then this is used to compute the probability of answering knowledge components correctly (the output).

As it was written in the paper: "However, since all of the information captured by the RNN lives in a same vector space in the hidden layer, the DKT model is consequently difficult to provide consistent prediction across time and therefore failing to pinpoint accurately which KCs a student is good at or unfamiliar with" 
Dynamic Key Value Memory Network(DKVMN) was proposed to solve this problem
In DKVMN we have two types of memory: static key memory and dynamic value memory. The keys can be thought as latent concepts while values as knowledge states. The vector representation of knowledge components and knowledge states are created and then they are used to predict P(a).DKVMN outperformed DKT.
Nevertheless, the vectors still lack the interpretability.   


Deep Learning Based Knowledge Tracing (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

In this paper authors propose transformer model  that would solve knowledge tracing problem. Student learning process largely depends on his/her past learning experience and the skills that (s) he acquired in the past while solving the tasks. SAKT(self attentive model fro knowledge tracing) firstly identifies KCs. In this papers exercises were used as KCs. Their model assigns weights for the previously  solved items and then predicts the probability of answering particular question correctly. Their model outperformed previous models by 4.43% gain in AUC score. 

Introduction (A Self-Attentive model for Knowledge Tracing)

1. Description
2. Embedding Layer
3. Feed Forward Layer
4. Residual Connection
5. Layer Normalization
6. Prediction Layer
7. Network Training

Proposed method  (A Self-Attentive model for Knowledge Tracing)

Experimental Setting (A Self-Attentive model for Knowledge Tracing)

1. Student Performance Prediction
2. Attention Weights Visualization
3. Ablation Study
4. Training Efficiency

Learn Before

Related

Learn After