Evaluating Model Generalization
Imagine a team has adapted a large, pre-trained language model using a dataset exclusively composed of instructions for summarizing legal documents and their corresponding correct summaries. The model becomes highly proficient at this specific task. Explain why this same adapted model would likely fail to generate a coherent and creative poem when given the instruction, 'Write a short poem about the ocean.'
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A development team fine-tunes a large, pre-trained language model using a high-quality dataset of 10,000 examples, all focused on converting natural language queries into structured database queries. The resulting model performs this specific task with near-perfect accuracy. However, when the team later attempts to use this same model for more general tasks like creative writing or summarizing news articles, they find its responses are often nonsensical or poorly structured. Which of the following statements best analyzes the most likely reason for this discrepancy in performance?
AI Model Generalization Failure
Evaluating Model Generalization