Learn Before
Determining Context Window Parameters
A language model's attention mechanism uses the notation { \mathbf{k}{i-n_c+1}, \dots, \mathbf{k}i } to represent the set of key vectors in its context window. If, at a particular step, the model is attending to the specific set { \mathbf{k}{21}, \mathbf{k}{22}, \mathbf{k}{23}, \mathbf{k}{24}, \mathbf{k}_{25} }, what are the numerical values for the current position i and the context window size n_c? Explain your reasoning.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An attention mechanism is processing a sequence of inputs. At the current position, indexed as
i=8, the model needs to attend to a context window of key vectors from the lastn_c=5positions (including the current position). Which of the following notations correctly represents this specific set of key vectors?A model is processing a sequence and is currently at position
i=10. It is using a context window of sizen_c=7. According to the notation { \mathbf{k}_{i-n_c+1}, \dots, \mathbf{k}_i }, the first key vector in this specific context window would be at index ____.Determining Context Window Parameters