Output Stage in Top-k Sampling
The output stage, labeled as step ④, is the final step in the top-k sampling process. In this stage, the single token that was chosen during the selection and sampling phase is finalized as the output. For example, after the token 'on' was sampled from the renormalized distribution, it becomes the definitive next token in the generated sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Expansion Stage in Top-k Sampling
Ranking and Pruning Stage in Top-k Sampling
A language model is generating the next word in a sentence and has calculated the probabilities for five potential words: 'house' (0.4), 'car' (0.3), 'boat' (0.15), 'plane' (0.1), and 'train' (0.05). The model uses a sampling method where it first ranks these words by probability, keeps only a specific number of the top-ranked words, renormalizes their probabilities to sum to 1, and then samples from this smaller set. How would decreasing the number of top-ranked words kept (e.g., from 4 to 2) most likely affect the generated text over time?
A language model is using a specific decoding method to generate the next token in a sequence. Arrange the following actions into the correct chronological order.
Ranking Stage in Top-k Sampling
Selection and Sampling Stage in Top-k Sampling
Output Stage in Top-k Sampling
Output Stage in Top-k Sampling
Applying a Probabilistic Filtering Method
Output Stage in Top-k Sampling
A language model is generating text and has narrowed down the next possible word to three candidates with the following initial probabilities: 'run' (0.5), 'walk' (0.2), and 'jog' (0.1). Based on the process where a final selection is made from a limited set of top candidates, which of the following actions will occur next?
Calculating Rescaled Probabilities in Text Generation
A language model is generating a sentence and has identified the three most likely next tokens as 'bright' (initial probability 0.5), 'sunny' (initial probability 0.3), and 'warm' (initial probability 0.1). According to the procedure where a final token is chosen from a limited set of top candidates, the model is guaranteed to select 'bright' as the next token after the probabilities are rescaled, because it has the highest probability.
Learn After
Language Model Token Selection
A language model has calculated the probabilities for the next possible tokens in a sequence. The five most likely tokens are: 'the' (0.4), 'a' (0.2), 'on' (0.1), 'in' (0.05), and 'at' (0.05). If the model uses a selection process where only the top 3 candidates are considered (k=3), what will be the new, renormalized probability distribution for the tokens that are ultimately sampled from?
A language model is generating text and has just identified the 'k' most likely next tokens from a large vocabulary. Arrange the following actions in the correct chronological order to complete the generation of the single next token.