An engineer is training a large language model and observes that after the initial phase, the training loss becomes highly unstable, fluctuating wildly and sometimes leading to numerical errors that stop the process. Lowering the learning rate provided some initial help but did not fully resolve the issue. Which of the following strategies, focusing on the data batching process, is a recognized practical method for stabilizing the remainder of the training run?

Rationale for Dynamic Batch Sizing

An engineer is training a large language model and observes that the training loss is stable. To accelerate the training process, the engineer decides to implement a schedule that progressively increases the batch size throughout the training run. This action is an appropriate application of this technique for the given situation.