Optimizing Student Model Training
Based on the scenario described below, identify the most likely imbalance in the student model's training process and propose a specific adjustment to remedy the issue. Justify your reasoning.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Combined Training Objective Formula for Knowledge Distillation
Dynamic Adjustment of the Knowledge Distillation Coefficient (λ)
Optimizing Student Model Training
When training a smaller 'student' model using a combined objective that learns from both a larger 'teacher' model and the ground-truth data, what is the primary role of the component that learns directly from the ground-truth data?
A student model is being trained using a combined objective that incorporates learning from both a larger 'teacher' model and the ground-truth data. Match each learning source with its primary contribution to the student model's training process.