Parameter Updates in Full Fine-Tuning
A pre-trained model consists of an encoder (the main body of the model) and a classifier head (the final output layer). When adapting this model to a new, specific task using the full fine-tuning method, which set(s) of parameters are updated? Explain the reasoning behind this approach.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Fine-Tuning Objective Function
A development team begins with a large language model pre-trained on a vast, general-purpose text corpus. Their objective is to adapt this model to classify customer support emails into specific categories: 'Billing Inquiry', 'Technical Support', and 'Product Feedback'. They have a curated dataset of 10,000 support emails, each correctly labeled with one of the categories. If the team employs a full fine-tuning strategy, which statement accurately describes the process they will follow?
Risk Assessment of Full Fine-Tuning
Parameter Updates in Full Fine-Tuning