Bayesian Based Knowledge Tracing model was created in 1990s. The main disadvantage of this model is that the assumptions that are made are unrealistic, for instance, it is assumed that every student and every question that belong to the same skill should be treated equally. 


San Diego State University

1. Bayesian Based Knowledge Tracing
2. Factory Analysis Based Knowledge Tracing
3. Deep Learning Based Knowledge Tracing

Knowledge Tracing (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

As it was written in the paper, the are two main approaches to modeling student learning namely: **knowledge tracing** and **factor analysis**.

**Knowledge tracing** as it can clear from the name of this approach, here we trace student's knowledge and we model the development of student knowledge to predict the sequence of answers. Bayesian Knowledge Tracing model is the most famous model of knowledge tracing. 

**Factor Analysis** compared to knowledge tracing doesn't care about the order of observations. 

1. **Item Response Theory** (IRT) is factor analysis model:

$$ P(Y_{s, j} = 1) = \sigma(\alpha_s -  \delta_j) $$

Here $\alpha_s$ denotes student's  ability and $\delta_j$ item difficulty. (it is assumed that student ability is unchangeable, it is fixed). At first glance this might seem very simple model but it is able to learn better the **Knowledge Tracing** architectures like Deep Knowledge Tracing.

2. **Multidimensional Item Response Theory** (MIRT) extension of IRT:

$$ P(Y_{s, j} = 1) = \sigma(<\alpha_s, \delta_j> + d_j) $$

Here $alpha_s$ and $\delta_j$ denote exactly the same with only one difference, here we have multidimensional vectors and $d_j$  captures the easiness of the item j. 

In recent works, student history was also used in factor analysis and the following models where created: **Additive Factor Model** (AFM) and **Performance Factor Analysis** (PFA). 

The formula for the AFM looks as:

$$P(Y_{s. j }= 1) = \sigma (\sum_{k \in KC(j)}\beta_k + \gamma_k \alpha_{s,k} )$$

here $beta$ denotes bias for skill, $\alpha$ is for the number of attempts and the $\gamma$ is  for the bias 

$$ P(Y_{s,j} = 1) = \sigma (\sum_{k \in KC(j)}\beta_k + \gamma_k c_{s,k}  + \rho_k f_{s,k})$$

c (f) is for the number of correct (wrong) answers.

Knowledge Tracing Machine (KTM) incorporates IRT, MIRT, AFM and PFA. The probability of correctness is expressed as:

$$ P(Y_t = 1) = \sigma (\mu + \sum_{i=1}^N w_i x_{t,i} + \sum_{1\le i \le l \le N } x_{t, i} x_{t, l} <v_i, v_t>)$$

(**NOTE** this model was proposed in the paper: Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)


Here $\mu$ is a global bias, N  -  the number of abstract features, ( for instance, item parameters)., $x_t$ - a sample gathering all features collected at time t: which student answers which item, and information regarding prior
attempts (x is a sparse vector), $w_i$is the bias  of feature i  (w is the vector of biases) and $v_i \in \R^d$  its embedding of dimension d. In the prediction will contribute only those $x_t$ that are greater than 0.  The features involved in a sample $x_i$ are typically in sparse number, so this probability can be computed efficiently. 

DASH(Difficulty Ability and Student History) bridges the gap between factor analysis and memory models. Here both learning and forgetting processes are taken into account.  

The authors plan to extend DASH so that it can account for both multiple skills tagging and memory decay.

This is the example of activation of a knowledge tracing machine:

Modeling Student Learning and Forgetting (DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills)



Bayesian Based Knowledge Tracing (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

These models are similar to Item Response Theory models but here we use skill-level parameters in order to predict the probability of answer question correctly. Learning Factory Analysis can be expressed as :

$$ P(\alpha) = \sigma (\theta  + \sum_{j \in skills} (\lambda_j N_j - \beta_j))$$

The parameters have the following meanings:

1. $\sigma$ - sigmoid function
2. $\theta$ - student ability
3. $\beta_j$ - item difficulty
4. $N_j$ - number of attempts (input)
5. $\lambda_j$ - learning rate for the skill j

There was an assumption that the focus on the student performance will solve knowledge tracing problem in a better manner than having higher sensitivity for the student ability. This lead to a new model which is called Performance Factory Analysis (PFA):

$$P(\alpha) = \sigma (\sum_j \alpha_j S_j + \rho_jF_j - \beta_j))$$

The parameters can have the following meanings:
1. $\alpha_jS_j$ + $\rho_jF_j$ -  student ability
2. $\alpha_j$ -  learning rate for the skill j when it is applied successfully
3. $\rho_j$ - learning rate for the skill j when it is applied unsuccessfully

These models require manual labeling of the skills and cannot deal with inherent dependency among skills. 




Factory Analysis Based Knowledge Tracing (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)


Deep learning based knowledge tracing(DKT), namely, modeling student knowledge state based on recurrent neural networks,  have shown better results than the BKT (Bayesian Knowledge Tracing) and PFA (Performance Knowledge Tracing).  (**NOTE** without human engineered features)
 
Here we transform fixed length inputs using  one-hot encoding and then they are passed to the hidden layer. The hidden state can be thought as a latent knowledge state of the student. Then this is used to compute the probability of answering knowledge components correctly (the output).

As it was written in the paper: "However, since all of the information captured by the RNN lives in a same vector space in the hidden layer, the DKT model is consequently difficult to provide consistent prediction across time and therefore failing to pinpoint accurately which KCs a student is good at or unfamiliar with" 
Dynamic Key Value Memory Network(DKVMN) was proposed to solve this problem
In DKVMN we have two types of memory: static key memory and dynamic value memory. The keys can be thought as latent concepts while values as knowledge states. The vector representation of knowledge components and knowledge states are created and then they are used to predict P(a).DKVMN outperformed DKT.
Nevertheless, the vectors still lack the interpretability.   


Learn Before

Related