Learn Before
Analyzing the Impact of 'k' on Token Pruning
A language model generates the following potential next tokens and their probabilities: {'for': 0.35, 'with': 0.25, 'in': 0.20, 'at': 0.15, 'on': 0.05}. Identify the set of tokens that would be discarded (pruned) if the filtering parameter 'k' is set to 2, and the set that would be discarded if 'k' is set to 4. Explain the relationship between the value of 'k' and the size of the discarded set.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model has generated the following potential next tokens and their associated probabilities:
{'the': 0.45, 'a': 0.25, 'my': 0.15, 'your': 0.10, 'his': 0.05}. If this list of tokens proceeds to a filtering stage where only the top 3 most probable tokens are kept for further processing, which set of tokens will be selected?Diagnosing Repetitive Text Generation
Analyzing the Impact of 'k' on Token Pruning