Learn Before
In a language model that combines its own predictions with information from retrieved examples using the formula Final_Prob = λ * Retrieved_Prob + (1 - λ) * Base_LM_Prob, setting the coefficient λ to 0 results in the final prediction being determined entirely by the retrieved examples.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Next Token Selection in k-NN Language Models
A language model's final probability for a word is determined by blending its own internal prediction with a prediction based on retrieved text examples. The formula used is:
Final_Prob = λ * Retrieved_Prob + (1 - λ) * Internal_Prob. In a scenario where the model's internal prediction for the next word is 'innovative', but the most frequent word in similar retrieved examples is 'creative', how would the value of the coefficientλinfluence the outcome?Analyzing Component Influence in a k-NN Language Model
In a language model that combines its own predictions with information from retrieved examples using the formula
Final_Prob = λ * Retrieved_Prob + (1 - λ) * Base_LM_Prob, setting the coefficientλto 0 results in the final prediction being determined entirely by the retrieved examples.