Calculating Renormalized Probability
A language model has predicted the probabilities for the five most likely next words in a sentence as follows:
- 'house': 0.35
- 'boat': 0.20
- 'car': 0.15
- 'train': 0.10
- 'plane': 0.05
If the model is configured to only consider the top 3 most probable words for its next selection, what is the new, recalculated probability for the word 'boat'? Show the formula or steps you used for your calculation.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model predicts the next token and assigns the following probabilities to the most likely candidates: 'the' (0.4), 'a' (0.2), 'one' (0.1), and 'some' (0.05). If the model is configured to only consider the top 3 most probable tokens for the next step, what is the adjusted probability for the token 'a' after the probabilities are recalculated to sum to 1?
Calculating Renormalized Probability
True or False: When a model identifies a small group of the most likely next words and then recalculates their probabilities so that they sum to 1, the new, recalculated probability for any given word in that group will always be greater than or equal to its original probability.