As it was written in the paper, the are two main approaches to modeling student learning namely: **knowledge tracing** and **factor analysis**.

**Knowledge tracing** as it can clear from the name of this approach, here we trace student's knowledge and we model the development of student knowledge to predict the sequence of answers. Bayesian Knowledge Tracing model is the most famous model of knowledge tracing. 

**Factor Analysis** compared to knowledge tracing doesn't care about the order of observations. 

1. **Item Response Theory** (IRT) is factor analysis model:

$$ P(Y_{s, j} = 1) = \sigma(\alpha_s -  \delta_j) $$

Here $\alpha_s$ denotes student's  ability and $\delta_j$ item difficulty. (it is assumed that student ability is unchangeable, it is fixed). At first glance this might seem very simple model but it is able to learn better the **Knowledge Tracing** architectures like Deep Knowledge Tracing.

2. **Multidimensional Item Response Theory** (MIRT) extension of IRT:

$$ P(Y_{s, j} = 1) = \sigma(<\alpha_s, \delta_j> + d_j) $$

Here $alpha_s$ and $\delta_j$ denote exactly the same with only one difference, here we have multidimensional vectors and $d_j$  captures the easiness of the item j. 

In recent works, student history was also used in factor analysis and the following models where created: **Additive Factor Model** (AFM) and **Performance Factor Analysis** (PFA). 

The formula for the AFM looks as:

$$P(Y_{s. j }= 1) = \sigma (\sum_{k \in KC(j)}\beta_k + \gamma_k \alpha_{s,k} )$$

here $beta$ denotes bias for skill, $\alpha$ is for the number of attempts and the $\gamma$ is  for the bias 

$$ P(Y_{s,j} = 1) = \sigma (\sum_{k \in KC(j)}\beta_k + \gamma_k c_{s,k}  + \rho_k f_{s,k})$$

c (f) is for the number of correct (wrong) answers.

Knowledge Tracing Machine (KTM) incorporates IRT, MIRT, AFM and PFA. The probability of correctness is expressed as:

$$ P(Y_t = 1) = \sigma (\mu + \sum_{i=1}^N w_i x_{t,i} + \sum_{1\le i \le l \le N } x_{t, i} x_{t, l} <v_i, v_t>)$$

(**NOTE** this model was proposed in the paper: Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)


Here $\mu$ is a global bias, N  -  the number of abstract features, ( for instance, item parameters)., $x_t$ - a sample gathering all features collected at time t: which student answers which item, and information regarding prior
attempts (x is a sparse vector), $w_i$is the bias  of feature i  (w is the vector of biases) and $v_i \in \R^d$  its embedding of dimension d. In the prediction will contribute only those $x_t$ that are greater than 0.  The features involved in a sample $x_i$ are typically in sparse number, so this probability can be computed efficiently. 

DASH(Difficulty Ability and Student History) bridges the gap between factor analysis and memory models. Here both learning and forgetting processes are taken into account.  

The authors plan to extend DASH so that it can account for both multiple skills tagging and memory decay.

This is the example of activation of a knowledge tracing machine:

Modeling Student Learning and Forgetting (DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills)



This is the way the authors encode feature to the vector x:

1. **Users** - they create one-hot encoders for each students.
2. **Items** - the questions/tasks/items are represented as one-hot encoders.
3. **Skills** - they  can be allocated extra s features for s skills. The skill that is included in the observation of a student over a question j are the ones of KC(j).
4. **Attempts** -  can be encoded of counters of the attempts to learn a skill.
5. **Wins & Fails** - they can be allocated as s extra features for successfully answering/solving an item and s extra features for the unsuccessful answers. 
6. **Extra side information** - the information like school id, teacher id, low stake or high stake practice can be concatenated to the other features.

This is the example of encoding users + items + skills + wins + fails:


Data and encoding of side information (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

Knowledge tracing machines involves IRT (item response theory), AFM (additive factor model), PFA (performance factory analysis).  
For the d = 0 (when we don't have embedding):

**Relation to IRT**  if all the features that we have are users, questions and pairs of user and answer are represented as one-hot encoders the expression for the knowledge tracing machines becomes:

$$ \log \frac {p(x)} {1 - p(x)} = \mu +  w_i + w_{n+j} = \theta_i - d_i $$

This means that the knowledge tracing machines become Rasch Model (1-PL IRT model).

**Relation to AFM and PFA** if we the features that we have are skills wins and fails at skill level and additionally suppose that q-matrix between questions and skills is also known. If w and x are encoded in the following way:
$(\beta_1, ... \beta_s, \lambda_1, ... \lambda_s, \delta_1, delta_s) $, $(q_{j1}, ... ,1_{js}, W_{i1}, ..., W_{is}, F_{i1}, F_{is})$ - (Ws, Fs  are counters for the successful /unsuccessful attempts)  then Knowledge tracing machines behave exactly like AFM and PFA.

**Relation to MIRT** if d > 0 then knowledge tracing machine acts like MIRT.


Relation to existing models (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

In training following function was minimized:

$$NLL(p(X), y) = \sum_{i=1}^S y_i \log p(x_i) + (1-y_i) \log (1-p(x_i))  $$
 here ys are outcomes and xs are inputs.

Biases, embeddings follow hyperpriors - $w_k \sim N ( \mu, 1 /\ \lambda)$
                                                                        $v_{kf} \ sim N ( \mu, 1 /\ \lambda)$
$\mu$, $\lambda$ follow hyperpriors -  $ \mu \sim N(0, 1)$
                                            $ \lambda \sim \Gamma (0, 1)$
  
Due to these hyperpriors the authors don't need finetuning.



Training (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

1. General
2. Data and encoding of side information
3. Relation to existing models
4. Training
5. Visualizing Embeddings

San Diego State University

1. Introduction
2. Related Work
3. Knowledge Tracing Machines
4. Experiments
5. Results and Discussion
6. Future Work

Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing

To improve learning processes on online educational platforms, accurately modeling student knowledge is essential. A significant challenge arises when trying to measure knowledge across students who have attempted different subsets of questions. Traditional student learning models often rely on sequence prediction or factor analysis, typically using unidimensional parameters for students or questions. The Knowledge Tracing Machines (KTM) framework addresses these limitations by generalizing these models to higher dimensions (up to twenty). Additionally, KTM leverages side information by identifying the specific knowledge components involved in each question—often structured as a Q-matrix—to better capture complex learning interactions.

Introduction (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

1. paper: https://arxiv.org/pdf/1811.03388.pdf

2. code: https://github.com/jilljenn/ktm

Reference for Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing

1. Knowledge Tracing
2. Factor Analysis
3. Item Response Theory
4. AFM(additive factory models) and PFA (performance factory analysis)
5. Factorization Machines

Related Work (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

Knowledge Tracing Machines (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

Experiments (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

On the assistments dataset KTM  training was much faster than DKT (deep knowledge tracing). It was observed that wins and fails (performance factory analysis) encoding improves performance a lot compared to encoding the number of attempts (additive factor model). When the bias was added for all the items the performance improved in the case of IRT (item response theory) but same can't be said about PFA (performance factory analysis). This is explained by the fact that the amount of items is large and adding difficulty parameter would be helpful in this case. For some of the datasets it was easy to identify skills but for others it was difficult which was caused by the fact that there were too few skills or only one KC was associated with the item. Increasing dimensions only slightly increased the performance when temporal datasets were used.  

Results and Discussion (Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing)

The authors made predictions in test set on totally new students. In this paper only pairwise interactions were provided and for the future research it would be interesting to generalize interactions to k-way terms. Instead of binary outcomes it might be interesting to consider graded outcomes.

Learn Before

Related

Learn After