Learn Before
Expansion Stage in Top-k Sampling
The expansion stage is the initial step in the top-k sampling process, as depicted in stage ① of the corresponding flowchart. Given a preceding context, such as the word 'is', the language model generates a set of all possible subsequent tokens from its vocabulary. In the provided example, this results in five candidate words: '.', 'cute', 'on', 'are', and 'sick'.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Expansion Stage in Top-k Sampling
Ranking and Pruning Stage in Top-k Sampling
A language model is generating the next word in a sentence and has calculated the probabilities for five potential words: 'house' (0.4), 'car' (0.3), 'boat' (0.15), 'plane' (0.1), and 'train' (0.05). The model uses a sampling method where it first ranks these words by probability, keeps only a specific number of the top-ranked words, renormalizes their probabilities to sum to 1, and then samples from this smaller set. How would decreasing the number of top-ranked words kept (e.g., from 4 to 2) most likely affect the generated text over time?
A language model is using a specific decoding method to generate the next token in a sequence. Arrange the following actions into the correct chronological order.
Ranking Stage in Top-k Sampling
Selection and Sampling Stage in Top-k Sampling
Output Stage in Top-k Sampling
Output Stage in Top-k Sampling
Applying a Probabilistic Filtering Method
Learn After
A language model is in the process of generating the next word after the context 'The ocean is'. The model's entire vocabulary is limited to these five words: ['deep', 'blue', 'cold', 'vast', 'empty']. In the very first step of its decision-making process, the model must generate a list of all possible candidates for the next word. What does this initial list of candidates look like?
During the very first step of generating the next word for a given text, a language model produces a candidate list that includes every single token from its entire vocabulary.
A developer is troubleshooting a text generation system. They provide the input context 'The sun is shining and the sky is'. In the very first step of generating the next word, the system produces a candidate list containing only {'blue', 'clear', 'bright'}. The system's full vocabulary, however, contains over 10,000 words. Based on this observation, which fundamental principle of this initial generation stage has been incorrectly implemented?