Learn Before
Evaluating Model Adaptation Strategies
A research lab has developed a powerful, general-purpose language model pre-trained on a massive corpus of text from the internet. They now want to adapt this model for a niche medical task: classifying patient notes into five distinct diagnostic categories. The lab has a high-quality, but very small, labeled dataset for this task and limited computational resources (i.e., budget for GPU time). They are debating two adaptation methods:
- Method A: Update all of the model's parameters using the small medical dataset.
- Method B: Keep the core language-understanding part of the model fixed, and only train the final classification part of the model on the medical dataset.
Which method would you recommend for the research lab? Justify your choice by evaluating the primary trade-offs of each method in the context of the lab's specific situation.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating Model Adaptation Strategies
A machine learning engineer is adapting a large pre-trained language model for a new text classification task. Due to limited computational resources, they decide to freeze the encoder's parameters and only train the new classifier head. What is the primary trade-off associated with this decision?
Parameter Optimization in Model Adaptation