Why Uneven Training Sizes Save Time
Question: Explain why learning curves are often measured using training set sizes that are spaced unevenly. Describe what makes this faster and when the idea is worth using.
Sample answer: Uneven spacing is used to reduce the amount of computation needed to build a learning curve. Small training runs finish quickly, but once the dataset becomes large, each additional model takes much longer to train. If you evaluate sizes in equal steps, you end up spending time on many expensive large runs that add only a little new information. Using a sequence that grows more quickly, such as 500, 1,000, 2,000, 4,000, and 8,000 examples, still shows the overall shape of the curve while avoiding some of the slow intermediate experiments. This approach is most useful when training each extra model is costly; if models train quickly, simple equal spacing is usually acceptable.
Key points:
- The main goal is to cut down total computation.
- Larger training sets take longer to train than smaller ones.
- Faster-growing size choices still reveal the overall curve.
- The method matters most when extra runs are expensive.
Rubric: The essay should identify reduced computation as the main reason for uneven spacing. It should explain that larger runs take longer, that skipping some intermediate large sizes saves time without losing the general trend, and that the method is only worthwhile when training cost is substantial.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why sample training set sizes at uneven intervals?
Smaller Training Sets Usually Train Faster
Reducing Cost with Nonlinear Data Sizes
Choosing Dataset Sizes for Learning Curves
Choosing Spacing for Learning-Curve Experiments
Why Uneven Training Sizes Save Time
Selecting Sample Sizes for an Efficient Learning Curve
When are uneven training-set sizes useful?
Why Use Uneven Training-Set Sizes for a Learning Curve?
You should always use evenly spaced training-set sizes, even when training extra models is expensive.