Learn Before
Comparing Training Objectives for Model Adaptation
A pre-trained language model is excellent at completing text passages but struggles to function as a helpful assistant that follows specific commands. Explain how the training objective used to adapt such a model to follow instructions differs from its original pre-training objective, and why this change is necessary.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A team is fine-tuning a pre-trained language model using a dataset of high-quality instruction-response pairs. The training process aims to adjust the model's parameters to maximize the probability of it generating the exact target response for each given instruction. After training, the team observes that the model often produces responses that are factually correct but much shorter and less detailed than the high-quality examples in their dataset. What is the most likely reason for this behavior, given the training objective?
Comparing Training Objectives for Model Adaptation
Adapting a General Model for a Specialized Task