Learn Before
Evaluating Model Generalization
A research team fine-tunes a pre-trained language model using a large dataset of question-answer pairs. The dataset exclusively contains questions that begin with the word 'What', such as 'What is the capital of France?' paired with 'Paris'. After training, the model is tested on the following three new prompts. Analyze the results and explain the most significant limitation demonstrated by the model's performance.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A team fine-tunes a pre-trained language model on a dataset consisting of 50,000 examples. Every example in the dataset follows the same format: an instruction to 'Summarize the following article,' followed by a long news article and a professionally written one-paragraph summary. After training, the model is tested with a new instruction: 'Summarize the following article,' followed by a news article it has never seen before. Which outcome would best demonstrate that the model has successfully generalized from its training?
Evaluating Model Generalization
Analyzing a Model's Performance Failure