Introduce weight matrices in the transformer

Generation of Query, Key, and Value Vectors in Self-Attention

In a self-attention mechanism, instead of directly comparing the raw input vectors of a sequence, each input vector is first multiplied by three separate, learned parameter matrices. This process creates three distinct representations of the original vector before they are used to calculate attention scores and output values. What is the primary analytical advantage of this approach over simply comparing the original input vectors to each other?

Value Weight Matrix $W_j^v$

Value Weight Matrix Definition ($\mathbf{W}_j^v \in \mathbb{R}^{d \times \frac{d}{\tau}}$)