Comparison of Training Objectives: Instruction Fine-Tuning vs. Pre-training
The training objective of instruction fine-tuning differs from that of standard language model pre-training. Instead of maximizing the probability of an entire sequence, instruction fine-tuning aims to maximize the conditional probability of generating the correct output (the remainder of the sequence) given a specific input prefix or instruction.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Parameter Initialization and Moderate Adjustment in Fine-Tuning
Probabilistic Objective of Supervised Fine-Tuning
Comparison of Training Objectives: Instruction Fine-Tuning vs. Pre-training
A machine learning team has successfully developed a large language model by training it on a massive, general-purpose text corpus. They now want to make the model better at following specific user commands. To do this, they have created a new, high-quality dataset that is much smaller than the original corpus and consists of example commands paired with ideal responses. Based on the standard procedures for adapting such models, which statement best describes the relationship between the initial training phase and this new adaptation phase?
Training Strategy for a Specialized Chatbot
The training methodology for instruction fine-tuning must be fundamentally different from the methodology used for pre-training, primarily because the dataset used for fine-tuning is substantially smaller.
Objective of Instruction Fine-Tuning
Learn After
A language model undergoes two distinct training stages. In the first stage, it is trained on a massive, unlabeled dataset of books and websites with the goal of learning to predict the next word in any given sentence. In the second stage, it is trained on a smaller, curated dataset of user prompts paired with ideal answers, with the goal of learning to generate helpful responses to the prompts. Which statement best analyzes the fundamental shift in the model's training objective between these two stages?
Differentiating Training Objectives in Language Models
During instruction fine-tuning, the model's training objective is to maximize the probability of the entire input-output sequence, treating the user's instruction and the desired response as a single, continuous piece of text.