The proposed model has similar structure as the transformer, consists of n stacked encoders and decoders. Encoder consists of self-attention and feedforward layers, after each of these layer there is a residual connection and layer normalization. Similarly, decoders contain the same layers and additionally fully connected layer (for prediction). Encoder and decoders have distinctive inputs, for the encoder we have $e_i$s, question metadata features and for the decoders we have response features shifted by one -> $(S, l_1, \cdots, l_{n-1})$ and the output of encoder. In their model, the authors use MultiHeadAttention and upper triangular masking.