Optimizing Cost in a Large Project
Case context: You are managing a machine learning project where training a model on 10,000 examples takes 10 hours. A junior engineer proposes generating a learning curve by training models on 1,000, 2,000, 3,000, 4,000, 5,000, 6,000, 7,000, 8,000, 9,000, and 10,000 examples.
Question: What should you diagnose about this proposed plan, and what alternative decision should you make based on the principles of learning curve efficiency?
Sample answer: I would diagnose that the junior engineer's plan is highly computationally expensive because it requires training many models on increasingly large, slow-to-train datasets (such as 7k, 8k, and 9k examples). Instead, I would decide to use nonlinearly spaced training-set sizes, such as 1,000, 2,000, 4,000, 6,000, and 10,000 examples. This decision significantly reduces total computation time while still giving a clear sense of the trends in the learning curve.
Key points:
- Diagnose the proposed plan as highly computationally expensive
- Propose an alternative using nonlinearly spaced sizes
- Justify that the alternative still shows clear trends
- Justify that it avoids the heavy cost of evenly spaced large datasets
Rubric: The response must recognize the high computational cost of the proposed linear plan and recommend a nonlinearly spaced alternative. It should justify the alternative by stating it avoids the cost of large intermediate sizes while still preserving clear learning curve trends.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why Use Nonlinearly Spaced Sizes
Training Speed and Dataset Size
Reducing Cost with _____ Sizes
Learning Curve Spacing Trade-offs
Deciding on Learning Curve Spacing
Rationale for Nonlinear Spacing
Optimizing Cost in a Large Project
Condition for Using Nonlinear Spacing
Maintaining Curve Information
Mandatory Use of Nonlinear Spacing