Learn Before
Ranking and Pruning Stage in Top-k Sampling
In the second stage of the Top-k sampling process, all candidate tokens from the expansion phase are sorted in descending order based on their probabilities. A parameter 'k' defines the number of top candidates to retain. For example, with k=3, the candidates 'cute' (Pr=0.34), 'on' (Pr=0.32), and 'sick' (Pr=0.21) are kept, while lower-ranked candidates like 'are' (Pr=0.12) and '.' (Pr=0.01) are pruned, or removed from further consideration.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Expansion Stage in Top-k Sampling
Ranking and Pruning Stage in Top-k Sampling
A language model is generating the next word in a sentence and has calculated the probabilities for five potential words: 'house' (0.4), 'car' (0.3), 'boat' (0.15), 'plane' (0.1), and 'train' (0.05). The model uses a sampling method where it first ranks these words by probability, keeps only a specific number of the top-ranked words, renormalizes their probabilities to sum to 1, and then samples from this smaller set. How would decreasing the number of top-ranked words kept (e.g., from 4 to 2) most likely affect the generated text over time?
A language model is using a specific decoding method to generate the next token in a sequence. Arrange the following actions into the correct chronological order.
Ranking Stage in Top-k Sampling
Selection and Sampling Stage in Top-k Sampling
Output Stage in Top-k Sampling
Output Stage in Top-k Sampling
Applying a Probabilistic Filtering Method
Learn After
A company wants its customer service chatbot, which is powered by a large language model, to provide real-time order tracking information to users. The model was not trained on this specific, dynamic data, and the company wants to avoid the cost and complexity of constantly retraining the model. Which of the following approaches is the best example of using an external tool to enhance the model's capabilities at the time of use?
A language model is generating the next word and has calculated the initial probabilities for five potential tokens: 'mat' (0.45), 'floor' (0.25), 'windowsill' (0.15), 'couch' (0.10), and 'table' (0.05). If the model is configured to only consider the top 3 most probable tokens for the next step, which set of tokens is kept after the ranking and pruning stage?
Analyzing Text Generation Behavior