Learn Before
Role of the New Value Vector in Inference
In the context of a transformer model generating text one token at a time, a new vector represented by the notation Vi' is created for the most recently generated token. Briefly explain what this vector represents and what happens to it immediately after it is computed.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
During the process of generating text one token at a time, a model computes a new vector denoted as Vi' for the most recent token. To efficiently generate the next token, what is the primary, immediate purpose of this Vi' vector?
Role of the New Value Vector in Inference
In a transformer model generating text token-by-token, the newly computed value vector for the current position i', denoted as Vi', is immediately weighted by attention scores to produce the output for that same position i'.