The proposed model has a similar structure to the transformer, consisting of $n$ stacked encoders and decoders. The encoder consists of self-attention and feedforward layers; after each of these layers, there is a residual connection and layer normalization. Similarly, decoders contain the same layers with an additional fully connected layer for prediction. The encoder and decoders have distinctive inputs: for the encoder we have $e_i$s (question metadata features), and for the decoders we have response features shifted by one ($(S, l_1, \cdots, l_{n-1})$) and the output of the encoder. In their model, the authors use MultiHeadAttention and upper triangular masking.