Activity (Process)

Selection and Sampling Stage in Top-p Sampling

This core stage of Top-p sampling involves three steps. First, the smallest set of top-ranked tokens whose cumulative probability meets or exceeds a threshold 'p' is selected (e.g., for p=0.6, 'cute' with Pr=.34 and 'on' with Pr=.32 are selected as their sum is 0.66). All other tokens are pruned. Second, the probabilities of the selected tokens are renormalized to sum to 1 (e.g., 'cute' becomes Pr=.51 and 'on' becomes Pr=.49). Finally, one token is randomly sampled from this new, smaller distribution.

Image 0

0

1

Updated 2025-10-10

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences