How to build a dev-set error curve from different training set sizes
Question: Explain how to construct a curve that shows dev-set error as the amount of training data changes. Describe how to train the models, what results to record, and how to place the values on the graph.
Sample answer: To build the curve, train the same learning algorithm multiple times using different training-set sizes. For example, with 2,000 training examples, you might fit separate models on 200, 400, 800, 1,200, and 2,000 examples. After each model is trained, evaluate it on the dev set and record the dev-set error. Then draw the graph with training-set size on the x-axis and dev-set error on the y-axis so you can see how performance changes as more data is used.
Key points:
- Train separate copies of the model on different training-set sizes.
- Measure dev-set error for each trained model.
- Plot training-set size on the x-axis and dev-set error on the y-axis.
Rubric: The answer must include: 1) training separate copies of the model on different training-set sizes, 2) measuring dev-set error for each copy, and 3) plotting dev-set error versus training-set size with the correct axes.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Using Uneven Training Sizes to Save Time on Learning Curves
What belongs on the vertical axis of a learning curve built from different training-set sizes?
A learning curve is built by training one model on the full training set and then reading off performance at different checkpoints during that same run.
To make a learning curve, you train _____ versions of the model on training sets of different sizes.
Match each learning-curve element to its job in the experiment.
Arrange the steps for building a learning curve from a labeled training pool of 800 examples.
Which procedure correctly builds a learning curve from a fixed pool of 1,200 labeled examples?
Using the Same Validation Set for Subsampled Training Runs
Learning-curve axes
Match each training subset description to its role on a learning curve built from 1,200 total examples.
Arrange the steps for building a learning curve from multiple training sizes
How to build a dev-set error curve from different training set sizes
Plan a learning-curve experiment with 800 labeled examples.
What metric goes on the y-axis of a learning curve built from different training-set sizes?