Learn Before
Risk of Overfitting and Catastrophic Forgetting in SFT
Supervised Fine-Tuning presents a paradoxical challenge: while it aims to incorporate new knowledge, the process of updating a pre-trained model can lead to the loss of its original knowledge. This issue, known as catastrophic forgetting, is particularly pronounced when an LLM is fine-tuned extensively on a large SFT dataset. Such intensive training can cause the model to overfit the new data, which not only harms its ability to generalize to unseen examples but can also erase foundational knowledge acquired during pre-training.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Learn After
Strategies to Mitigate Overfitting and Catastrophic Forgetting in SFT
Fine-Tuning Performance Degradation
A team fine-tunes a large, pre-trained language model, known for its strong general knowledge, on a highly specialized dataset of legal contracts. They train the model for a very large number of iterations. After fine-tuning, the model demonstrates exceptional performance in generating and interpreting legal text but now provides nonsensical or incorrect answers to simple, general knowledge questions it could easily answer before. What is the most likely explanation for this change in the model's behavior?
The Interplay of Overfitting and Knowledge Loss in Model Tuning