Google

Plotting learning curves can be computationally expensive because it may require training many models at different data sizes. When that cost matters, using nonlinearly spaced training-set sizes, such as 1,000, 2,000, 4,000, 6,000, and 10,000 examples, can still provide a clear sense of the curve trends while avoiding evenly spaced sizes.

Nonlinear Training Set Sizes for Cheaper Learning Curves

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Why Use Nonlinearly Spaced Sizes

Training Speed and Dataset Size

Reducing Cost with _____ Sizes

Learning Curve Spacing Trade-offs

Deciding on Learning Curve Spacing

Question:
Explain the rationale behind using nonlinearly spaced training set sizes when constructing a learning curve. Discuss the underlying reason for the computational savings and state the condition under which this technique should be applied.

Sample answer:
The rationale for using nonlinearly spaced training set sizes is to reduce overall computational expense. Training models on large datasets takes significantly more time than training on small ones. If sizes are spaced linearly (e.g., every 1,000 examples up to 10,000), you are forced to train many large, slow models. By using nonlinear spacing (like 1k, 2k, 4k, 6k, 10k), you still obtain a clear sense of the learning curve's trends but can skip several intermediate large dataset sizes, saving significant time. This technique should only be applied if the computational cost of training all the additional models is significant; if training is very fast, linear spacing is fine.

Key points:
- Reduces computational cost
- Training small datasets is much faster than large datasets
- Still provides a clear sense of learning curve trends
- Relevant only if computational cost is significant

Rubric:
The essay should correctly identify reducing computational cost as the primary goal. It must explain that smaller datasets train much faster, and that skipping some evenly spaced larger sizes saves time while preserving the curve's trend. It must explicitly state that the technique is only relevant when training costs are significant.

Rationale for Nonlinear Spacing

Case context:
You are managing a machine learning project where training a model on 10,000 examples takes 10 hours. A junior engineer proposes generating a learning curve by training models on 1,000, 2,000, 3,000, 4,000, 5,000, 6,000, 7,000, 8,000, 9,000, and 10,000 examples.

Question:
What should you diagnose about this proposed plan, and what alternative decision should you make based on the principles of learning curve efficiency?

Sample answer:
I would diagnose that the junior engineer's plan is highly computationally expensive because it requires training many models on increasingly large, slow-to-train datasets (such as 7k, 8k, and 9k examples). Instead, I would decide to use nonlinearly spaced training-set sizes, such as 1,000, 2,000, 4,000, 6,000, and 10,000 examples. This decision significantly reduces total computation time while still giving a clear sense of the trends in the learning curve.

Key points:
- Diagnose the proposed plan as highly computationally expensive
- Propose an alternative using nonlinearly spaced sizes
- Justify that the alternative still shows clear trends
- Justify that it avoids the heavy cost of evenly spaced large datasets

Rubric:
The response must recognize the high computational cost of the proposed linear plan and recommend a nonlinearly spaced alternative. It should justify the alternative by stating it avoids the cost of large intermediate sizes while still preserving clear learning curve trends.

Optimizing Cost in a Large Project

Question:
Under what specific condition is the technique of using nonlinearly spaced training set sizes relevant?

Sample answer:
This technique is relevant only if the computational cost of training all the additional models for a standard linear curve is significant.

Key points:
- Relevant only if computational cost is significant

Rubric:
The answer must explicitly state that the technique is relevant when the computational cost of training models is significant.

Learn Before

Related