Comparison of SFT and Pre-training Datasets
The dataset used for Supervised Fine-Tuning (SFT) differs significantly from the one used for pre-training. While the SFT dataset is considerably smaller in volume, its content is highly specialized and curated for the specific tasks the model is being adapted for.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Instruction Fine-Tuning
Potential for Undesirable Content Generation After SFT
Example of SFT: Question-Answering Task
Applicability of Supervised Fine-Tuning
Practical Implementation Challenges of SFT
Maximum Likelihood Estimation (MLE) as the Objective for Supervised Fine-Tuning
Instruction Fine-Tuning as a Technique of SFT
Size and Specialization of SFT Datasets
Generalization as an Outcome of SFT
Characteristics of SFT Datasets
Generalization from Supervised Fine-Tuning
Definition of SFT Datasets
A development team starts with a base language model that has been pre-trained on a massive, general-purpose dataset from the web. To make the model a specialized customer service chatbot, the team initiates a second phase of training. How would the dataset used in this second phase most likely differ from the original pre-training dataset?
Comparison of SFT and Pre-training Datasets
SFT as a Post-Training Phase
Adapting a Model for a New Task
A law firm wants to develop a language model that can take a lengthy legal contract as input and produce a concise, one-paragraph summary highlighting key clauses like the term, liability limits, and governing law. They have a team of paralegals available to create a high-quality dataset of several thousand contract-summary pairs. Which of the following approaches is the most effective and direct way to train the model for this specific task?
Learn After
Dataset Selection for a Specialized AI Assistant
A development team has a large language model that was initially trained on a vast and diverse collection of text from the internet, enabling it to understand grammar, facts, and reasoning. The team's new goal is to adapt this model to become a specialized and helpful assistant for writing professional business emails. Which of the following data strategies for this second phase of training would be most effective in achieving this specific goal?
A large language model undergoes two main training stages, each using a different type of dataset. Match each characteristic below to the training stage dataset it best describes.