Attention Head Output with Grouped Queries and Causal Masking

Attention Head Output in Grouped-Query Attention (GQA)

A computational model processes sequences and, at a specific step i, maintains a collection of data represented as:

$\{ (\mathbf{K}_{\le i}^{[1]}, \mathbf{V}_{\le i}^{[1]}), \dots, (\mathbf{K}_{\le i}^{[ au]}, \mathbf{V}_{\le i}^{[ au]}) \}$

In this set, each $(\mathbf{K}, \mathbf{V})$ is a pair of matrices, the subscript ${\le i}$ indicates that the matrices contain information for all sequence positions from the start up to position i, and the superscript [t] is an index ranging from 1 to τ.

Based on this structure, which statement provides the most accurate analysis of the collection?

Interpreting a Set of Indexed Key-Value Pairs

State of Key-Value Cache During Generation