Learn Before
Output Stage in Top-p Sampling
The final stage of the Top-p sampling process is the output. The single token that was chosen via sampling from the renormalized probability distribution in the previous stage is presented as the final result. In the provided flowchart, the token 'on' is the final output of the process.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Ranking Stage in Top-p Sampling
Selection and Sampling Stage in Top-p Sampling
Output Stage in Top-p Sampling
Expansion Stage in Top-p Sampling
A language model is generating text and has calculated the probabilities for the following potential next tokens:
mat(0.5),floor(0.3),rug(0.1), andtable(0.05). The model is configured to use a sampling method where it first identifies the smallest set of the most probable tokens whose cumulative probability is at least 0.9. It then discards all other tokens and randomly selects the final output from this reduced set. Based on this process, what is the outcome?A language model is using a probabilistic method to generate the next word in a sentence. Arrange the following descriptions of the steps involved in this method into the correct chronological order.
Applying Probabilistic Text Generation
Learn After
A text generation system follows a multi-step procedure to select the next word. First, it identifies all potential words and their initial probabilities. Second, it sorts these words by probability and forms a small candidate group from the most likely options. Third, it recalculates the probabilities for only the words within this small group. Finally, it randomly picks one word from this group based on their recalculated probabilities. Which of the following represents the final output of this entire procedure?
In a text generation process that uses a cumulative probability threshold to select a group of potential next words, the final output consists of the entire group of selected words.
A text generation system is determining the next word after the phrase 'The ocean is'. After identifying and ranking all possible next words, it creates a small candidate group and recalculates their probabilities for the final selection step. The final candidate group and their probabilities are:
{'deep': 0.5, 'blue': 0.4, 'vast': 0.1}. Based on this information, what will be the output of the process?