The authors used four different datasets to test their models:

1. ASSISTments 2009-2010 - collected by the ASSISTments online tutoring system.

2. ASSISTments 2015 - larger dataset than the ASSISTments 2009-2010 dataset.

3. OLI Engineering Statics 2011 - college level statistics course dataset.
 
4. Synthetic-5 - dataset was synthesized by the authors of DKT paper. 

Dataset (Knowledge Query Network for Knowledge Tracing)

1. Correctness Prediction
Every dataset was split into train-valid-test sets except Synthetic-5. Number of epochs varied between 50-200 and they used grid search for the hyper-parameters. 
Hidden state size for RNN  and MLP layers and size of embedding dimensionality- 32, 64, 128.
Adam optimization was used for the error minimization. 
Apart from that, the authors fed KQN learned skill vectors to DKT model to evaluate their quality in predicting. They denote this model as DKT + KQN and the authors searched for the parameters in the same manner as in the KQN model.

2. Skill Domain Analysis
They clustered skills based on skill distances. After clustering they evaluated its performance by comparing them to ground true labels. 

3. Sensitivity Analysis of Vector Space Dimensionality
d - denotes dimensionality and $d_{opt}$ - optimal values of d which we get from the previous correctness prediction task. In order to measure the effect of d on the performance, the authors kept all the hyperparameters set to optimal values. 



Knowledge and Implementation (Knowledge Query Network for Knowledge Tracing)

San Diego State University

1. Introduction
2. Related Work
3. Our Proposed Model
4. Probabilistic Skill Similarity
5. Experiments
6. Results and Analysis
7. Conclusions and Future Work


Knowledge Query Network for Knowledge Tracing

There are two types of a learner models:

1. Domain Model
This types of models try to associate skills to a particular task and find out the structure of skill domain. 

2. Knowledge Model
This types of model observe how the student knowledge changes while he/she is practicing. This models are also referred as Knowledge Tracing models. In order KT models to have a good performance it is quintessential for them to apt in describing knowledge state of the student and to be able to correctly classify whether student will answer certain problem correctly.

As it was described in the paper current KT models lack the ability to describe knowledge state or knowledge interaction (interaction of student knowledge state at time t and skill at time t+1). In order to solve these issues the authors create a new model called Knowledge Query Network (KQN)  



Introduction (Knowledge Query Network for Knowledge Tracing)

1. Bayesian Based Knowledge Tracing
2. Factory Analysis Based Knowledge Tracing
3. Deep Learning Based Knowledge Tracing

Knowledge Tracing (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

1. Motivation
2. Objective
3. Architecture Overview
4. Inputs
5. Knowledge State Query
6. Knowledge State Encoder
7. Skill Encoder
8. Optimization

Our Propose Model (Knowledge Query Network for Knowledge Tracing)

1. Distance measures for skill vectors:

$$d_{Euclidean} (s_1, s_2) ^2 = ||s_1 - s_2||^2 = 2(1- s_1\cdot s_2) = 2d_{cosine}(s_1, s_2) , \forall s_1, s_2 \in U^d $$

2. Distances and Odds Ratios

Then the authors "show how a pairwise distance between two skill vectors is related to the logarithm of their odds ratio":
$$p = P( c = 1 | KS, s), o =\frac {p} {1-p}$$ 
$$ \log (\frac {o_1}{o_2} ) ^2 = (KS \cdot \delta_{1, 2} )^2 \times 2dcosine(s_1, s_2)$$
$$\delta_{1, 2} = \frac {s_1 - s_2} {|| s_1 - s_2||}$$

Probabilistic Skill Similarity (Knowledge Query Network for Knowledge Tracing)

Experiments (Knowledge Query Network for Knowledge Tracing)

Paper: https://arxiv.org/pdf/1908.02146.pdf

Code: https://github.com/JSLBen/Knowledge-Query-Network-for-Knowledge-Tracing

Reference for Knowledge Query Network for Knowledge Tracing

1. Correctness Prediction
2. Knowledge Interaction Visualization
3. Skill Domain Analysis
4. Sensitivity Analysis of the Vector Space Dimensionality

Results and Analysis (Knowledge Query Network for Knowledge Tracing)

1. KQN performs better than all the previous models.

2. KQN enables to query student knowledge state with respect to different skills.

3. KQN architecture leads to the concept of probabilistic skill similarity which is helpful for the domain modeling and clustering.

4. KQN is resistant towards changes in the variability of the dimensionality of the vector space for knowledge state and skill vectors.

KQN has a lot more parameters than neural networks and it is computationally heavier than the DKT. 
As it was mentioned in the paper, skill distances are "useful features for domain
modeling to explore the latent structure of the skill domain, which can be a future direction of the KT research"

Learn Before

Related

Learn After