An attention mechanism is processing the 10th element of a sequence (where the current index i=10) and is configured with a context window size of 5 (n_c=5). Based on the standard formula for constructing a memory component from a fixed-size window, Mem = (K_[i-n_c+1,i], V_[i-n_c+1,i]), which set of key vectors (represented by their indices) would be included in the key matrix K at this step?

In an attention mechanism using a fixed-size window, the memory component at step i is constructed using the formula Mem = (K_[i-n_c+1,i], V_[i-n_c+1,i]), where n_c is the context window size. What is the direct consequence of increasing the value of n_c?

Constant Memory Size in Fixed-Window Attention