Cross-entropy error and error terms are calculated: $$E(\theta_{model} | c_{t+1}, p_{t+1}) = - [ c_{t+1} \log p_{t+1} + (1-c_{t+1}) \log (1-p_{t+1})]$$ $$E_{total} (\theta_{model} | c_{2:t+1}, p_{2:t+1}) = \sum_{t+1} ^{T-1} E(\theta_{model} | c_{t+1}, p_{t+1})$$

Optimization (Knowledge Query Network for Knowledge Tracing)

The objective of the knowledge tracing model is to find the parameters of the Bernoulli distribution for correctness:
$$p_{t+1} = P(c_{t + 1} = 1| e_{1: t + 1}, c_{1:t+1})$$
$$c_{t+1} \sim Bernoulli (p_{t+1})$$
where $$e_t \in \{1, \dots, N\}$$ is the skill and $$c_t$$ is the correctness at time step $$t$$.

San Diego State University

Google

1. Motivation
2. Objective
3. Architecture Overview
4. Inputs
5. Knowledge State Query
6. Knowledge State Encoder
7. Skill Encoder
8. Optimization

Our Propose Model (Knowledge Query Network for Knowledge Tracing)

The authors of this paper aim to  generalize knowledge state as well as describe knowledge interaction. In order to do so, they project knowledge states and skill vectors into the same vector space.  Since knowledge state and skill vectors lie into the same vector space, the dot product between  knowledge state vector (at time t) and  skill vector (at time t+1) is taken in order to output the probability of correctness.

They limit the skill vectors to be positive orthant and  have unit-length to make learning one skill independent from learning another and make logit only dependent on the direction of the related skill vector. 

Motivation (Knowledge Query Network for Knowledge Tracing)

1. Inputs
For input we have two a student response at time t and skill at time t + 1. These responses are one-hot encoded and for RNN:
$$ x_t \in {0, 1}^{2N}$$
$$ x_t^k = 1$$ (wrong answer)
$$ x_t^{k + N} = 1$$ (correct answer)

N - amount of skills, k - skill at time t, $k^`$  is skill at time t+1 that is one-hot encoded to $e_{t+1}$ in which $k^`$ -th element is set to 1 and all other elements are 0.

2. Knowledge State Query
$$ y_{t+1} = KS_t \cdot s_{t+1}$$
$$ p_{t+1} = \sigma (y_{t+1})$$
KS is a knowledge state vector and s is a skill vectors. 
If the skills are independent than they will be orthogonal and they wouldn't have an effect on each other, otherwise increase/decrease in one vector would lead to increase/decrease in another.
The KQN approximates parameter of Bernoulli distribution as:

$$P (c_{t+1} = 1| e_{1:t+1}, c_{1:t}) = P(c_{t+1} = 1 | x_{1:t}, e_{t+1}) \approx \sigma(y_{t+1})$$ 

Inputs and Knowledge State Query (Knowledge Query Network for Knowledge Tracing)

1. **Knowledge State Encoder**: The knowledge state vector is generated as:

$$h_t = RNN(x_t)$$

$$KS_t = W_{h, KS} \cdot h_t + b_{h, KS}$$

LSTMs and GRUs were used as RNN variants.

2. **Skill Encoder**: $$e_{t+1}$$ is embedded to $$s_{t+1}$$ as:

$$ o_{t+1} = ReLU (W_1 \cdot (ReLU(W_0 \cdot e_{t+1} + b_0 )) + b_1)$$

$$ s_{t+1} = L2-normalize(o_{t+1})$$

$$ s_{t+1} \in U^d = \{ v \in \mathbb{R}^d : ||v|| = 1, v^i \ge 0 \quad i = 1, \dots , d \}$$

Knowledge State Encoder and Skill Encoder (Knowledge Query Network for Knowledge Tracing)

The Knowledge Query Network architecture consists of three components:
1. **Knowledge Encoder**: Student responses are converted into a knowledge state.
2. **Skill Encoder**: Skills are converted into skill vectors.
3. **Knowledge State Query**: The knowledge state and skill vectors are used to predict the probability of correctness.

Learn Before

Related

Learn After