Effect of Temperature Scaling on a Reward-Modified Distribution
An AI text generation model adjusts its output probabilities using the formula: Final_Prob = Ref_Prob * exp((1/β) * Reward), where 'Ref_Prob' is the initial probability from a base model, 'Reward' is a score for a specific quality (e.g., factual accuracy), and 'β' is a positive temperature parameter. A developer decreases the value of 'β' and observes that the model's outputs now adhere much more strictly to the rewarded quality, but have become less diverse and creative. Explain the mathematical reason for this change in the model's behavior.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An AI text generation system adjusts the likelihood of different outputs using the formula: New_Likelihood = Base_Likelihood * exp((1/β) * Reward). In this formula, 'Base_Likelihood' is the initial probability from a reference model, 'Reward' is a score for the output's quality, and 'β' is a positive 'temperature' parameter. A team wants to use this system to generate a diverse set of creative, high-quality story endings. They are comparing two settings for the temperature parameter: β = 0.5 and β = 2.0. Which setting should they choose to better achieve their goal, and why?
Tuning a Generative Model for Different Tasks
Effect of Temperature Scaling on a Reward-Modified Distribution