Learn Before
Top-p (Nucleus) Sampling Process
Top-p, or nucleus, sampling is a probabilistic text generation technique that involves a multi-stage process. First, in the expansion stage, all potential next tokens are generated. Second, these tokens are ranked by probability. Third, a 'nucleus' of the top-ranked tokens is selected, such that their cumulative probability exceeds a predefined threshold 'p'. The probabilities within this nucleus are then renormalized. Finally, a single token is sampled from this renormalized set to become the output. This method balances quality and diversity by filtering out the long tail of low-probability tokens.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Learn After
Ranking Stage in Top-p Sampling
Selection and Sampling Stage in Top-p Sampling
Output Stage in Top-p Sampling
Expansion Stage in Top-p Sampling
A language model is generating text and has calculated the probabilities for the following potential next tokens:
mat(0.5),floor(0.3),rug(0.1), andtable(0.05). The model is configured to use a sampling method where it first identifies the smallest set of the most probable tokens whose cumulative probability is at least 0.9. It then discards all other tokens and randomly selects the final output from this reduced set. Based on this process, what is the outcome?A language model is using a probabilistic method to generate the next word in a sentence. Arrange the following descriptions of the steps involved in this method into the correct chronological order.
Applying Probabilistic Text Generation