Full-batch gradient descent computes each parameter update using every training example in the dataset. Because the gradient is based on all available data, the update direction is stable and low-noise, but each step can be slow and memory-intensive when the dataset is large. Online gradient descent uses one training example per update. This makes each step cheap and light on memory, but the gradient estimate is very noisy, so the training path can bounce around instead of moving smoothly toward the minimum. Mini-batch gradient descent uses a small group of examples for each update, choosing a batch size between 1 and the full dataset size. This often gives the best practical tradeoff: more stable updates than online training and much better hardware efficiency than full-batch processing. Although online updates can look efficient in terms of examples processed, the per-example computation is not well suited to modern vectorized hardware. Mini-batches such as 64 or 128 examples often achieve the best wall-clock speed because they make better use of optimized linear algebra routines.