Learn Before
Objective of Supervised Fine-Tuning
The primary goal of Supervised Fine-Tuning (SFT) is to adapt a model that already has pre-trained parameters, denoted as . The adaptation process involves adjusting these parameters to maximize the conditional probability of generating the desired output sequence, , given a specific input sequence, . This objective, expressed as maximizing , aligns the model's behavior with the provided instruction-response examples.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Computational Expense of SFT for Large Language Models
Objective of Supervised Fine-Tuning
Computational Efficiency of Fine-Tuning Compared to Pre-training
Suitability of Fine-Tuning for Aligning with Human Values
Definition of LLM Alignment
Supervised Fine-Tuning for LLM Alignment
A company has a powerful, general-purpose language model that can write essays, answer questions, and summarize articles. They want to adapt this model to perform a new, specialized task: generating concise and helpful summaries of customer support tickets. Which of the following strategies represents the most direct and effective approach to adapt the model's internal parameters for this specific purpose?
Designing a Dataset for Model Behavior Adaptation
Embedding Task Knowledge into LLM Parameters via Fine-Tuning
Supervised Fine-Tuning (SFT) as an Example of Labeled Data Fine-Tuning
Diagnosing Unintended Model Behavior After Adaptation
Learn After
Maximum Likelihood Estimation (MLE) as the Objective for Supervised Fine-Tuning
A development team is fine-tuning a pre-trained language model using a curated dataset of customer support inquiries (inputs) and their corresponding ideal, human-written responses (outputs). The aim is to create a specialized chatbot that reliably provides answers in the same helpful and accurate style as the examples. From a probabilistic perspective, which statement best describes the fundamental objective of this training process?
Correcting a Flawed Fine-Tuning Objective
Objective for a Specialized Math Tutor
Mathematical Formulation of the Supervised Fine-Tuning Objective
Conditional vs. Joint Probability Objectives in Language Modeling