1Cademy - Set of Superscript-Indexed Vectors ($$\{ \mathbf{q}_{i}^{[1]}, \dots, \mathbf{q}

Formula

Set of Superscript-Indexed Vectors ( $\{ \mathbf{q}_{i}^{[1]}, \dots, \mathbf{q}_{i}^{[\tau]} \}$ )

The notation $\left\{ \mathbf{q}_{i}^{[1]}, \dots, \mathbf{q}_{i}^{[\tau]} \right\}$ represents a set of vectors. Each vector, denoted by $\mathbf{q}_i$ , is further identified by a superscript index enclosed in square brackets, such as $[1]$ or $[\tau]$ . This indexing scheme is commonly used to represent a collection of vectors associated with a single element $i$ across multiple parallel contexts, for instance, the query vectors for the $i$ -th token across $\tau$ attention heads.

Updated 2026-07-03

Contributors are:

Who are from:

References

Reference of Foundations of Large Language Models Course

Learn After

A computational model analyzes the 4th element in a sequence. To do this, it generates 12 distinct 'probe' vectors for this single element, one for each of its 12 parallel processing modules. Which of the following notations correctly represents the complete set of all 'probe' vectors for the 4th element?
True or False: In the notation { \mathbf{q}{i}^{[1]}, \dots, \mathbf{q}{i}^{[ au]} }, the subscript i represents one of \tau parallel contexts, while the superscript (e.g., [1]) refers to a specific element in a sequence.