Learn Before
Selection and Sampling Stage in Top-p Sampling
This core stage of Top-p sampling involves three steps. First, the smallest set of top-ranked tokens whose cumulative probability meets or exceeds a threshold 'p' is selected (e.g., for p=0.6, 'cute' with Pr=.34 and 'on' with Pr=.32 are selected as their sum is 0.66). All other tokens are pruned. Second, the probabilities of the selected tokens are renormalized to sum to 1 (e.g., 'cute' becomes Pr=.51 and 'on' becomes Pr=.49). Finally, one token is randomly sampled from this new, smaller distribution.

0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Ranking Stage in Top-p Sampling
Selection and Sampling Stage in Top-p Sampling
Output Stage in Top-p Sampling
Expansion Stage in Top-p Sampling
A language model is generating text and has calculated the probabilities for the following potential next tokens:
mat(0.5),floor(0.3),rug(0.1), andtable(0.05). The model is configured to use a sampling method where it first identifies the smallest set of the most probable tokens whose cumulative probability is at least 0.9. It then discards all other tokens and randomly selects the final output from this reduced set. Based on this process, what is the outcome?A language model is using a probabilistic method to generate the next word in a sentence. Arrange the following descriptions of the steps involved in this method into the correct chronological order.
Applying Probabilistic Text Generation
Learn After
A text generation model uses a method where it selects the smallest set of most likely next words whose combined probability is at least a certain threshold, renormalizes their probabilities to sum to 1, and then samples from that new set. Given the following sorted list of potential next words and their initial probabilities, and a probability threshold of 0.7, which option correctly shows the set of words and their new probabilities after selection and renormalization, just before the final sampling step?
Initial Probabilities:
- "the": 0.40
- "a": 0.20
- "an": 0.15
- "one": 0.10
- "some": 0.08
A language model is configured to generate the next word in a sentence using a specific text generation technique. This technique involves identifying a core set of the most probable words based on a cumulative probability threshold, adjusting their probabilities, and then making a final selection. Arrange the following actions into the correct sequence as they would occur within this specific stage of the generation process.
Analyzing Generation Outcomes