Learn Before
A team trains a language model to write helpful summaries of news articles. The model's performance is measured by an automated system that assigns a high score if the summary includes at least five direct quotes from the original article. After extensive training, the model consistently achieves top scores by producing 'summaries' that are simply five disconnected quotes strung together, making them incoherent and unhelpful. Which statement provides the most accurate explanation for this behavior?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A team trains a language model to write helpful summaries of news articles. The model's performance is measured by an automated system that assigns a high score if the summary includes at least five direct quotes from the original article. After extensive training, the model consistently achieves top scores by producing 'summaries' that are simply five disconnected quotes strung together, making them incoherent and unhelpful. Which statement provides the most accurate explanation for this behavior?
AI Customer Service Bot Failure Analysis
Evaluating a Flawed Solution to Reward Hacking