Cross-entropy error and error terms are calculated:

$$E(\theta_{model} | c_{t+1}, p_{t+1}) = - [ c_{t+1} \log p_{t+1} + (1-c_{t+1}) \log (1-p_{t+1})]$$

$$E_{total} (\theta_{model} | c_{2:t+1}, p_{2:t+1}) = \sum_{t+1} ^{T-1} E(\theta_{model} | c_{t+1}, p_{t+1})

San Diego State University

1. Motivation
2. Objective
3. Architecture Overview
4. Inputs
5. Knowledge State Query
6. Knowledge State Encoder
7. Skill Encoder
8. Optimization

Our Propose Model (Knowledge Query Network for Knowledge Tracing)

The authors of this paper aim to  generalize knowledge state as well as describe knowledge interaction. In order to do so, they project knowledge states and skill vectors into the same vector space.  Since knowledge state and skill vectors lie into the same vector space, the dot product between  knowledge state vector (at time t) and  skill vector (at time t+1) is taken in order to output the probability of correctness.

They limit the skill vectors to be positive orthant and  have unit-length to make learning one skill independent from learning another and make logit only dependent on the direction of the related skill vector. 

Motivation (Knowledge Query Network for Knowledge Tracing)

1. Objective
Knowledge tracing model tries to find parameters of the Bernoulli distribution: 
$$ p_{t+1} = P(c_{t + 1} = 1| e_{1: t + 1}, c_{1:t+1})$$
$$ c_{t+1} \sim Bernoulli (p_{t+1})$$
$e_t \in {1, ... N}$ is skill, $c_{t}$ is correctness at time step t.

2. Architecture

Knowledge Query Network consists of 3 components:

1. Knowledge Encoder
The student responses are converted in knowledge state.

2.  Skill Encoder
Skill is converted into skill vector.

3. Knowledge State Query
These two vectors are passed to knowledge state query to predict the probability of correctness.

Objective and Architecture Overview (Knowledge Query Network for Knowledge Tracing)

1. Inputs
For input we have two a student response at time t and skill at time t + 1. These responses are one-hot encoded and for RNN:
$$ x_t \in {0, 1}^{2N}$$
$$ x_t^k = 1$$ (wrong answer)
$$ x_t^{k + N} = 1$$ (correct answer)

N - amount of skills, k - skill at time t, $k^`$  is skill at time t+1 that is one-hot encoded to $e_{t+1}$ in which $k^`$ -th element is set to 1 and all other elements are 0.

2. Knowledge State Query
$$ y_{t+1} = KS_t \cdot s_{t+1}$$
$$ p_{t+1} = \sigma (y_{t+1})$$
KS is a knowledge state vector and s is a skill vectors. 
If the skills are independent than they will be orthogonal and they wouldn't have an effect on each other, otherwise increase/decrease in one vector would lead to increase/decrease in another.
The KQN approximates parameter of Bernoulli distribution as:

$$P (c_{t+1} = 1| e_{1:t+1}, c_{1:t}) = P(c_{t+1} = 1 | x_{1:t}, e_{t+1}) \approx \sigma(y_{t+1})$$ 

Inputs and Knowledge State Query (Knowledge Query Network for Knowledge Tracing)

1.  Knowledge State Encoder
Knowledge state vector is generated as:
$$h_t = RNN(x_t)$$
$$KS_t = W_{h, KS} \cdot h_t + b_{h, KS}$$

LSTMs and GRUs were used as RNN variants.

2. Skill Encoder
$e_{t+1}$ is embedded to $s_{t+1}$ as:
$$ o_{t+1} = ReLU (W_1 \cdot (ReLU(W_0 \cdot e_{t+1} + b_0 )) + b_1)$$
$$ s_{t+1} = L2-normalize(o_{t+1})$$
$$ s_{t+1} \in U ^d = { v\in \R^d : ||v|| = 1, v^i \ge 0 \quad i = 1,  ... , d}$$


Learn Before

Related