Fitting LLM Learning Curves with Diverse Functions

A research team is modeling the performance of a large language model as they increase the amount of training data. Their predictive model, based on a standard power-law function, anticipates a steady, continuous improvement in performance. However, their experiments show that the model's error rate first decreases, then temporarily increases, before decreasing again. Which statement best analyzes the limitation of their predictive model in this context?

Evaluating a Predictive Model for LLM Training

Predicting Complex Learning Dynamics