Generalization from Supervised Fine-Tuning
A key outcome of Supervised Fine-Tuning (SFT) is the model's ability to generalize its learning. After being trained on a set of specific examples, such as question-answer pairs, the LLM can correctly respond to new questions that were not included in the SFT dataset.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Instruction Fine-Tuning
Potential for Undesirable Content Generation After SFT
Example of SFT: Question-Answering Task
Applicability of Supervised Fine-Tuning
Practical Implementation Challenges of SFT
Maximum Likelihood Estimation (MLE) as the Objective for Supervised Fine-Tuning
Instruction Fine-Tuning as a Technique of SFT
Size and Specialization of SFT Datasets
Generalization as an Outcome of SFT
Characteristics of SFT Datasets
Generalization from Supervised Fine-Tuning
Definition of SFT Datasets
A development team starts with a base language model that has been pre-trained on a massive, general-purpose dataset from the web. To make the model a specialized customer service chatbot, the team initiates a second phase of training. How would the dataset used in this second phase most likely differ from the original pre-training dataset?
Comparison of SFT and Pre-training Datasets
SFT as a Post-Training Phase
Adapting a Model for a New Task
A law firm wants to develop a language model that can take a lengthy legal contract as input and produce a concise, one-paragraph summary highlighting key clauses like the term, liability limits, and governing law. They have a team of paralegals available to create a high-quality dataset of several thousand contract-summary pairs. Which of the following approaches is the most effective and direct way to train the model for this specific task?
Learn After
A team fine-tunes a pre-trained language model on a dataset consisting of 50,000 examples. Every example in the dataset follows the same format: an instruction to 'Summarize the following article,' followed by a long news article and a professionally written one-paragraph summary. After training, the model is tested with a new instruction: 'Summarize the following article,' followed by a news article it has never seen before. Which outcome would best demonstrate that the model has successfully generalized from its training?
Evaluating Model Generalization
Analyzing a Model's Performance Failure