1Cademy - Set of Indexed Key-Value Pairs

Learn Before

Set of Key-Value Pairs
Set of Sequential Key-Value Pairs

Formula

Set of Indexed Key-Value Pairs

A set representing a sequence of Key-Value pairs, indexed from 1 to $\tau$ . Each element in the set is a tuple $(\mathbf{K}_{\le i}^{[t]}, \mathbf{V}_{\le i}^{[t]})$ , where $t$ is the index (e.g., representing a layer or head). The bold letters $\mathbf{K}$ and $\mathbf{V}$ typically denote Key and Value matrices or vectors, and the subscript ${\le i}$ indicates that these keys and values are for all positions up to and including $i$ in a sequence. The full expression is given by: $\{ (\mathbf{K}_{\le i}^{[1]}, \mathbf{V}_{\le i}^{[1]}), \dots, (\mathbf{K}_{\le i}^{[\tau]}, \mathbf{V}_{\le i}^{[\tau]}) \}$ .