Stacked Layer Architecture and Final Output in Transformers

Formula for Single-Head Self-Attention

Within a single layer of a Transformer model during inference, a sequence of input vectors is processed through a two-step sequence. Which statement best analyzes the distinct roles of the self-attention mechanism and the subsequent Feed-Forward Network (FFN) in this process?

Arrange the following computational steps in the correct order as they occur within a single layer of a Transformer model during inference.

Debugging a Transformer Layer