Risk Assessment of Full Fine-Tuning
A research lab starts with a large language model pre-trained on a diverse corpus of web text. Their goal is to adapt this model for a highly specialized task: identifying specific genetic mutations mentioned in biomedical research papers. For this new task, they only have a small, carefully labeled dataset of 500 examples. If the lab applies the full fine-tuning method, where all of the model's parameters are updated, what is a significant risk they face regarding the model's performance, and why does this risk arise in this specific scenario?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Fine-Tuning Objective Function
A development team begins with a large language model pre-trained on a vast, general-purpose text corpus. Their objective is to adapt this model to classify customer support emails into specific categories: 'Billing Inquiry', 'Technical Support', and 'Product Feedback'. They have a curated dataset of 10,000 support emails, each correctly labeled with one of the categories. If the team employs a full fine-tuning strategy, which statement accurately describes the process they will follow?
Risk Assessment of Full Fine-Tuning
Parameter Updates in Full Fine-Tuning