1. Knowledge State Encoder: The knowledge state vector is generated as:

$h_t = RNN(x_t)$

$KS_t = W_{h, KS} \cdot h_t + b_{h, KS}$

LSTMs and GRUs were used as RNN variants.

2. Skill Encoder: $e_{t+1}$ is embedded to $s_{t+1}$ as:

$o_{t+1} = ReLU (W_1 \cdot (ReLU(W_0 \cdot e_{t+1} + b_0 )) + b_1)$

$s_{t+1} = L2-normalize(o_{t+1})$

$s_{t+1} \in U^d = \{ v \in \mathbb{R}^d : ||v|| = 1, v^i \ge 0 \quad i = 1, \dots , d \}$