A team is training a language model to generate helpful and coherent paragraphs. They are comparing two feedback strategies:
- Strategy A: An automated system provides a small reward after every 5 words are generated, based on whether those words match a predefined vocabulary list.
- Strategy B: A human expert reads the entire completed paragraph and provides a single, holistic quality score.
Based on principles of effective training for complex language tasks, which strategy is likely to produce a better model, and why?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A team is training a language model to generate helpful and coherent paragraphs. They are comparing two feedback strategies:
- Strategy A: An automated system provides a small reward after every 5 words are generated, based on whether those words match a predefined vocabulary list.
- Strategy B: A human expert reads the entire completed paragraph and provides a single, holistic quality score.
Based on principles of effective training for complex language tasks, which strategy is likely to produce a better model, and why?
Evaluating Feedback Mechanisms for AI Training
In the context of training a large language model to generate creative stories, increasing the frequency of automated reward signals (e.g., a reward for every grammatically correct sentence) is always more effective than providing a single, holistic quality rating from a human expert at the end of the entire story.
Optimizing Feedback for an Empathetic AI