Parameter Matrices for Attention Transformations

Introduce weight matrices in the transformer

Calculating an Output Vector in a Simple Sequence Model

In a simple self-attention mechanism where similarity is measured by dot product and weights are normalized by a softmax function, if a current input vector x_i is perfectly orthogonal to a preceding input vector x_j, then x_j will have zero influence on the final output vector y_i.

You are calculating the output vector y_i for a single input vector x_i in a sequence using a simple self-attention mechanism that only considers preceding elements. Arrange the following computational steps in the correct chronological order.