Learn Before
Diagnosing and Correcting LLM Behavior
A development team is building a chatbot using a large language model that was only pre-trained on a vast corpus of internet text. When they give the model the prompt, 'Instruction: Translate the following English sentence to French. Input: I would like to buy a coffee.', the model responds with, 'This is a common phrase used by tourists when visiting Paris, and learning it can be very helpful.'
Evaluate the model's response. What core issue does this behavior reveal about the model's training, and what specific tuning process should be implemented to make the model correctly perform the requested task? Justify your answer.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Instruction-Following Ability in LLMs
Supervised Fine-Tuning (SFT)
Instruction Data Generation and Collection
Generalization in Instruction Alignment
Suitability of Instruction Fine-Tuning for Well-Defined Tasks
An AI developer provides the exact same input to two different large language models. Model A is a base model trained solely to predict the next word in a sequence. Model B is the same base model but has undergone an additional tuning process.
Input given to both models: "Instruction: Summarize the following paragraph in exactly one sentence. Paragraph: The process of photosynthesis allows plants to convert light energy into chemical energy. This chemical energy is stored in the form of glucose, which serves as the primary source of food for the plant. During this process, carbon dioxide is absorbed from the atmosphere and oxygen is released as a byproduct, which is essential for most life on Earth."
Model A's Output: "This process is crucial for maintaining the balance of gases in our planet's atmosphere and provides the foundation for nearly all terrestrial ecosystems."
Model B's Output: "Photosynthesis is the process where plants use light energy to create their own food, converting carbon dioxide into oxygen as a byproduct."
Based on these outputs, which statement provides the most accurate analysis of the models' behaviors?
Diagnosing and Correcting LLM Behavior
Supervised Fine-Tuning (SFT) as an Example of Labeled Data Fine-Tuning
An AI development team is creating a dataset to fine-tune a pre-trained language model, aiming to improve its ability to follow user commands. Which of the following instruction-response pairs represents the highest-quality data point for this specific purpose?