Causal Attention Weight Matrix Calculation

An attention mechanism processes the input sequence: ['The', 'robot', 'grasped', 'the', 'wrench']. The attention weight matrix is calculated to determine the contextual importance of each word. The row in the matrix corresponding to the word 'grasped' has the highest weight value in the column corresponding to the word 'wrench'. What does this high weight signify?

Interpreting an Attention Weight Matrix

In an attention mechanism processing a sequence of m items, an m x m attention weight matrix is generated. What does the i-th row of this matrix fundamentally represent?

Query-Key-Value Attention Output Matrix Product