Learn Before
Evaluating Pre-training Scenarios
Consider two scenarios:
Scenario A: A team is building a system to classify legal documents into one of 500 highly specific, proprietary categories. They have a massive, well-labeled dataset of 10 million documents.
Scenario B: A team is building a chatbot to answer questions about a new software product. They have a small, curated dataset of only 500 question-and-answer pairs.
Which scenario would benefit more from starting with a large, general-purpose pre-trained model? Justify your answer by explaining the relationship between the amount of available task-specific data and the primary advantage of the pre-training approach.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A startup is developing a chatbot to answer questions about a new, highly specialized medical device. They have a very small, curated dataset of only a few hundred question-and-answer pairs. Which of the following best explains the primary advantage of using a large, pre-trained model as their starting point?
Evaluating Pre-training Scenarios
Evaluating Model Training Strategies for a Low-Resource Task