Learn Before
Iterative Context Update in Autoregressive Generation
In autoregressive text generation, the process is iterative. After a set of one or more tokens is predicted in a single step, these new tokens are appended to the existing context. This newly expanded context is then used as the basis for the next prediction cycle, allowing the model to continue generating the sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Token Selection from Probability Distribution
Step-by-Step Example of Auto-Regressive Sequence Generation
Mathematical Formulation of Draft Model Prediction in Speculative Decoding
Iterative Context Update in Autoregressive Generation
Key-Value (KV) Cache in Transformer Inference
Sequential Generation of Output Tokens
Context Shifting in Auto-Regressive Generation
A language model is generating a sentence and has so far produced the sequence:
['The', 'cat', 'sat']. Based on the principles of sequential, one-at-a-time token generation where each new token depends on the ones before it, what is the direct input the model will use to determine the next token in the sequence?A language model generates text by producing a single token at each step, using the entire sequence generated so far as the context for the next token. Arrange the following events in the correct chronological order to illustrate the generation of two new tokens following the initial input 'The ocean is'.
A researcher develops a novel text generation model. Given an input like 'The movie was', instead of generating one token at a time, this model predicts the entire completion (e.g., 'incredibly boring and predictable') in a single, parallel step. Which core principle of the standard auto-regressive process is fundamentally violated by this new model's design?
Learn After
An autoregressive language model is generating text. It has been provided the initial context 'The sun is shining and the'. In its first step, it predicts the next word will be 'sky'. To generate the word that comes after 'sky', what sequence will the model use as its new input?
An autoregressive language model is given the initial context 'The old clock ticked'. It then generates the next two tokens, which are the word 'loudly' followed by a period '.'. Arrange the following four actions into the correct chronological sequence that describes this two-step generation process.
Diagnosing a Repetitive Generation Loop