Vector Slice Notation for a Sequence Window ($\mathbf{V}_{[i-n+1,i]}$)

Matrix Slice Notation for a Sequence Window ($\mathbf{K}_{[i-n+1,i]}$)

A process is analyzing a sequence of data points, [10, 20, 30, 40, 50, 60, 70, 80, 90, 100], which are indexed starting from 0. The process needs to select a window of n=5 consecutive elements. If the last element of the window is at index i=7, which of the following sub-sequences represents the correct window of data?

Debugging a Sequence Slicing Function

An algorithm processes a long sequence of data by examining a 'window' of n=10 consecutive elements at a time. If the algorithm is currently focused on the specific window of elements from index 90 to index 99 (inclusive), then the variable i representing the end-point of the current window would be 100.

Set of Indexed Key-Value Pairs

Set of Superscript-Indexed Vectors

Set of Key-Value Pairs

Function of a Sequence of Overlined Variables

Function of a Sequence of Averaged Vectors

Vector Slice Notation for a Sequence Window ($\mathbf{V}_{[i-n+1,i]}$)

Set of Sequential Vectors Notation

Vector Sequence Window Notation

Consider an autoregressive model generating a sequence of tokens one by one. At each step i, the model calculates attention using the query from the current token and the keys and values from all tokens generated so far (from position 1 to i). To optimize this process, the model maintains a growing set of all previously computed key and value vectors. What is the primary computational advantage of this strategy?

State of an Autoregressive Cache

An autoregressive language model with τ parallel computational units (e.g., attention heads) is generating a sequence of tokens. After computing the output for the 3rd token, the model stores the key and value vectors from all tokens processed so far to use in subsequent steps. Which of the following notations correctly represents the complete set of these stored key-value pairs at this specific moment?