Concept shift, where the definitions of target labels change over time, is challenging to fix in a principled manner. While extreme shifts might require collecting new labels and training from scratch, such shifts are rare in practice. Instead, tasks usually change gradually. To correct for slow concept shift, the standard approach is to use the existing neural network weights and perform a few update steps with the newly collected data, effectively fine-tuning the model to adapt to the recent changes.