An engineer is designing a self-attention layer for a text processing model. They notice that as they increase the dimensionality (d_k) of the query and key vectors, the training process becomes unstable, and the gradients used for learning become extremely small. Which of the following best explains this phenomenon and the standard solution implemented within the attention mechanism?

Calculating a Scaled Attention Score

A transformer's self-attention layer calculates an output vector for each input token. Arrange the following computational steps in the correct sequence to produce a single output vector, based on its query vector and the full set of key and value vectors for the input sequence.

Attention Score in Transformers ($\beta_{i,j}$)