Learn Before
Generalization in Instruction Alignment
A significant challenge within instruction alignment is achieving generalization, which refers to a model's ability to correctly follow new instructions that were not part of its fine-tuning dataset. The ultimate goal is for the model to understand and execute a wide range of commands, rather than merely memorizing the specific examples it was trained on.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Instruction-Following Ability in LLMs
Supervised Fine-Tuning (SFT)
Instruction Data Generation and Collection
Generalization in Instruction Alignment
Suitability of Instruction Fine-Tuning for Well-Defined Tasks
An AI developer provides the exact same input to two different large language models. Model A is a base model trained solely to predict the next word in a sequence. Model B is the same base model but has undergone an additional tuning process.
Input given to both models: "Instruction: Summarize the following paragraph in exactly one sentence. Paragraph: The process of photosynthesis allows plants to convert light energy into chemical energy. This chemical energy is stored in the form of glu
Diagnosing and Correcting LLM Behavior
Supervised Fine-Tuning (SFT) as an Example of Labeled Data Fine-Tuning
An AI development team is creating a dataset to fine-tune a pre-trained language model, aiming to improve its ability to follow user commands. Which of the following instruction-response pairs represents the highest-quality data point for this specific purpose?
Learn After
Language Model Performance Analysis
An AI development team fine-tunes two language models. Model A is trained on 100,000 examples of a single, narrow task: rephrasing sentences into five specific styles. Model B is trained on 10,000 examples covering a wide variety of tasks (e.g., summarization, translation, creative writing). When both models are tested on a completely new, unseen instruction like 'generate a grocery list for a three-course Italian meal,' which outcome is most likely?
Evaluating Training Data Strategies for Model Performance