Strategies to Mitigate Overfitting and Catastrophic Forgetting in SFT
Several strategies can be employed to mitigate overfitting and catastrophic forgetting during Supervised Fine-Tuning. Common techniques include using regularization and early stopping, applying a smaller learning rate to make gentle adjustments to the model's weights, and incorporating data from diverse sources and problem domains to improve robustness.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Strategies to Mitigate Overfitting and Catastrophic Forgetting in SFT
Fine-Tuning Performance Degradation
A team fine-tunes a large, pre-trained language model, known for its strong general knowledge, on a highly specialized dataset of legal contracts. They train the model for a very large number of iterations. After fine-tuning, the model demonstrates exceptional performance in generating and interpreting legal text but now provides nonsensical or incorrect answers to simple, general knowledge questions it could easily answer before. What is the most likely explanation for this change in the model's behavior?
The Interplay of Overfitting and Knowledge Loss in Model Tuning
Learn After
Diagnosing and Correcting a Fine-Tuning Process
Evaluating a Flawed Fine-Tuning Strategy
A development team is fine-tuning a large, pre-trained language model on a specialized dataset of medical research papers. They observe that while the model's performance on medical queries is excellent, it has started to perform poorly on simple, general-knowledge questions it could previously answer correctly. Which of the following adjustments to the fine-tuning process is the most direct and effective strategy to address this degradation in general capabilities?
A machine learning engineer is fine-tuning a pre-trained language model and observes several undesirable behaviors. Match each observed behavior with the most appropriate mitigation strategy.