The authors used four different datasets to test their models:

1. ASSISTments 2009-2010 - collected by the ASSISTments online tutoring system.

2. ASSISTments 2015 - larger dataset than the ASSISTments 2009-2010 dataset.

3. OLI Engineering Statics 2011 - college level statistics course dataset.
 
4. Synthetic-5 - dataset was synthesized by the authors of DKT paper. 

San Diego State University

Experiments (Knowledge Query Network for Knowledge Tracing)

Dataset (Knowledge Query Network for Knowledge Tracing)

1. Correctness Prediction
Every dataset was split into train-valid-test sets except Synthetic-5. Number of epochs varied between 50-200 and they used grid search for the hyper-parameters. 
Hidden state size for RNN  and MLP layers and size of embedding dimensionality- 32, 64, 128.
Adam optimization was used for the error minimization. 
Apart from that, the authors fed KQN learned skill vectors to DKT model to evaluate their quality in predicting. They denote this model as DKT + KQN and the authors searched for the parameters in the same manner as in the KQN model.

2. Skill Domain Analysis
They clustered skills based on skill distances. After clustering they evaluated its performance by comparing them to ground true labels. 

3. Sensitivity Analysis of Vector Space Dimensionality
d - denotes dimensionality and $d_{opt}$ - optimal values of d which we get from the previous correctness prediction task. In order to measure the effect of d on the performance, the authors kept all the hyperparameters set to optimal values. 



Learn Before

Related