Tuning a Generative Model for Different Tasks
Based on the case study below, analyze the engineer's problem. For each application (Medical Q&A and Marketing Slogans), determine whether a high or low value for the parameter β would be more appropriate and explain your reasoning by referencing how it impacts the overall formula.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An AI text generation system adjusts the likelihood of different outputs using the formula: New_Likelihood = Base_Likelihood * exp((1/β) * Reward). In this formula, 'Base_Likelihood' is the initial probability from a reference model, 'Reward' is a score for the output's quality, and 'β' is a positive 'temperature' parameter. A team wants to use this system to generate a diverse set of creative, high-quality story endings. They are comparing two settings for the temperature parameter: β = 0.5 and β = 2.0. Which setting should they choose to better achieve their goal, and why?
Tuning a Generative Model for Different Tasks
Effect of Temperature Scaling on a Reward-Modified Distribution