In their model the authors used scaled dot product attention mechanism. The layer tries to find corresponding weight to each exercise  in order to correctly measure whether student will answer the question correctly or not.  They used multiple heads in order to gather the information from different subspaces. They use only first t interactions in order to predict t+1 interaction.

San Diego State University

1. Description
2. Embedding Layer
3. Feed Forward Layer
4. Residual Connection
5. Layer Normalization
6. Prediction Layer
7. Network Training

Proposed method  (A Self-Attentive model for Knowledge Tracing)

Model predicts whether student will answer the question correctly or not based on the previous sequence of interactions. The problem is transformed into sequential modeling problem. Interaction pairs are represented as a number, namely:

$$y_t = e_t + r_t \times E$$
$e_t$ - exercise, $r_t$ - answer, E - total number of exercises. 

These are notations:

Description (A Self-Attentive model for Knowledge Tracing)

Input sequence is transformed so that they would have the same fixed size n. If the size of sample is less than n than padding is applied otherwise inputs are divided and separate samples are generated. Then the interaction embedding matrix is trained  $M \in \R^{2E \times d}$ which is used to get $s_i$s. Exercise embedding matrix is trained in a similar manner $E \in \R^{E \times d}$. Position encoding is used in the model in order to encode the order of sequence. Position embedding is learned in the process of training $P \in R^{n \times d}$. The $P_i$s are added to the interaction embedding matrix.  Therefore as a result of embedding layer embedded interaction and exercise matrices are generated. 

Learn Before

Related