Learn Before
Selection and Sampling Stage in Top-k Sampling
This stage, labeled as step ③, follows the ranking and pruning phase in Top-k sampling. It involves two key actions on the selected top-k candidates. First, the probabilities of these candidates are rescaled, or 'renormalized,' so that they sum to 1. For example, the initial probabilities of 'cute' (0.34), 'on' (0.32), and 'sick' (0.21) are adjusted to new values of 0.39, 0.36, and 0.25, respectively. Second, a single token is chosen by randomly sampling from this new, smaller probability distribution. In the example, the token 'on' is selected through this sampling process.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Expansion Stage in Top-k Sampling
Ranking and Pruning Stage in Top-k Sampling
A language model is generating the next word in a sentence and has calculated the probabilities for five potential words: 'house' (0.4), 'car' (0.3), 'boat' (0.15), 'plane' (0.1), and 'train' (0.05). The model uses a sampling method where it first ranks these words by probability, keeps only a specific number of the top-ranked words, renormalizes their probabilities to sum to 1, and then samples from this smaller set. How would decreasing the number of top-ranked words kept (e.g., from 4 to 2) most likely affect the generated text over time?
A language model is using a specific decoding method to generate the next token in a sequence. Arrange the following actions into the correct chronological order.
Ranking Stage in Top-k Sampling
Selection and Sampling Stage in Top-k Sampling
Output Stage in Top-k Sampling
Output Stage in Top-k Sampling
Applying a Probabilistic Filtering Method
Learn After
Output Stage in Top-k Sampling
A language model is generating text and has narrowed down the next possible word to three candidates with the following initial probabilities: 'run' (0.5), 'walk' (0.2), and 'jog' (0.1). Based on the process where a final selection is made from a limited set of top candidates, which of the following actions will occur next?
Calculating Rescaled Probabilities in Text Generation
A language model is generating a sentence and has identified the three most likely next tokens as 'bright' (initial probability 0.5), 'sunny' (initial probability 0.3), and 'warm' (initial probability 0.1). According to the procedure where a final token is chosen from a limited set of top candidates, the model is guaranteed to select 'bright' as the next token after the probabilities are rescaled, because it has the highest probability.