For sequence length $n$ and representation dimension $d$, a self-attention layer requires $O(n^2 \cdot d)$ computation and $O(1)$ sequential operations. A standard recurrent layer requires $O(n \cdot d^2)$ computation and $O(n)$ sequential operations because each hidden state $h_t$ depends on $h_{t-1}$. Thus, when $n < d$, as is typical for word-piece or byte-pair sequences in the supplied course context, self-attention has lower asymptotic per-layer complexity and permits parallel computation across sequence positions. For long sequences, restricting each position to an attention neighborhood of size $r$ reduces the per-layer complexity to $O(r \cdot n \cdot d)$ while retaining $O(1)$ sequential operations.