Learn Before
Calculating the Interpolation Coefficient
A language model combines a base distribution and a retrieval-based distribution to predict the next token. The model ultimately selects the token 'cat' with a final combined probability of 0.55. If the base model assigned 'cat' a probability of 0.40, and the retrieval-based model assigned it a probability of 0.80, what was the interpolation coefficient (位) used to balance the two distributions? Provide the final value.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model combines a base probability distribution, P_lm, with a retrieval-based distribution, P_knn, to predict the next token. The final probability is calculated by blending these two distributions using an interpolation coefficient, 位 = 0.6. Given the distributions below for a small vocabulary, which token will the model select as its final output?
- P_lm: {'wordA': 0.5, 'wordB': 0.4, 'wordC': 0.1}
- P_knn: {'wordA': 0.2, 'wordB': 0.7, 'wordC': 0.1}
Analyzing Conflicting Model Predictions
Calculating the Interpolation Coefficient