Deep knowledge tracing (DKT), deep item response theory (Deep-IRT), deep key value memory netword (DKVMN) models have relatively similar results here is the table that shows their performances:


San Diego State University

Experiments (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

To test their models, they used 5 different datasets:

1. **ASSIST2009**  - " The dataset contains 4, 151 students with 325, 637 question and answering interactions from 26, 688 questions of 110 skills."

2. **ASSIST2015** - "This dataset contains 19, 840 student responses for 100  skills with a total of 683, 801 question-and-answering interactions"

3. **STATIC2011** - "This dataset is obtained from an engineering statics course with 189, 927 interactions from 333 students and 1, 223 question tags of 156 skills"

4. **SYNTHETIC** - 2000 students question-and-answer trajectories were synthesized.

5. **FSAI-F1toF3** - "It consists of 51, 283 question-and-answer interactions from 310 students on 2, 266 questions of 99 skills"

Dataset (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

1.  Experimental Setting

Input ($q_t$, $a_t$) are fed using their ID tags: $ID(q_t) \in {1, ... , Q}$ and $ID(q_t, a_t) = q_t + a_t Q \in {1, ... , 2Q} $ 
All the parameters (KC matrix A, $M^k$, $M^v$, W, b)  are initialized randomly from Gaussian distribution and they are learned during training so that the cross-entropy loss is minimized:
$$  L = - \sum_t (a_t \log p_t + (1 - a) \log (1-p_t) )$$ 
Adam optimizer was used where learning rate was set to 0.003 and the batch size was 32.  The timestaps have fixed length 200 except Synthetic dataset.

2. Hyperparameter Selection and Evaluation

30% of the dataset is used for the test set and rest is used for the training. 5-fold cross-validation is used for hyperparameter selection. Hidden layer size, the dimensions for key and value matrices are chosen from 10, 50, 100, 200. 
They use ROC curve to measure the performance of their model.  



Learn Before

Related