Learn Before
Output Stage in Top-k Sampling
The output stage is the final step (labeled ④) in a single iteration of the Top-k sampling process. The token that was selected through sampling in the previous stage is confirmed as the generated output. This token is then appended to the sequence of text, and the model can begin the entire process again to generate the next token.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Expansion Stage in Top-k Sampling
Ranking and Pruning Stage in Top-k Sampling
A language model is generating the next word in a sentence and has calculated the probabilities for five potential words: 'house' (0.4), 'car' (0.3), 'boat' (0.15), 'plane' (0.1), and 'train' (0.05). The model uses a sampling method where it first ranks these words by probability, keeps only a specific number of the top-ranked words, renormalizes their probabilities to sum to 1, and then samples from this smaller set. How would decreasing the number of top-ranked words kept (e.g., from 4 to 2) most likely affect the generated text over time?
A language model is using a specific decoding method to generate the next token in a sequence. Arrange the following actions into the correct chronological order.
Ranking Stage in Top-k Sampling
Selection and Sampling Stage in Top-k Sampling
Output Stage in Top-k Sampling
Output Stage in Top-k Sampling
Applying a Probabilistic Filtering Method
Learn After
A text generation model is in the process of creating a sentence. It has already produced the sequence 'The sun shines brightly in the'. In the step just completed, the model considered a small, filtered set of the most probable next words and, through a sampling process, selected the token 'sky'. What is the immediate next action the model must perform to continue the generation process?
A text generation model using a specific sampling strategy follows a multi-step process to produce one word at a time. Arrange the following descriptions of these steps into the correct logical order for a single generation cycle.
Finalizing a Token Generation Step