Using a Validation Curve to Judge the Value of More Data
Question: In a short analytical response, explain how a target error level and a plotted dev-loss trend can be used to judge whether collecting additional training examples is worthwhile.
Sample answer: Mark the desired error level on the graph, then extend the dev-loss trend beyond the observed points to estimate how much extra data might be needed to approach that level. This gives a rough planning estimate, not a promise of what will happen. In the example, the trend suggested that a larger training set, roughly about twice as large, could be enough to get near the goal.
Key points:
- Mark the goal level on the plot.
- Extend the dev-loss trend beyond the current data.
- Use the projection to estimate the possible gain from more examples.
- Treat the result as an approximation, not a certainty.
- Note that the example pointed to roughly doubling the training set.
Rubric: A strong response states how to place the target on the curve, describes extending the dev-loss trend to estimate the effect of added data, and explains that the conclusion is only an estimate rather than a guarantee.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does sketching an extension of the dev-error curve help you estimate?
A trend line from past data can give an exact guarantee of the model’s future error after more training data is added.
Add the target _____ to a dev-error curve before estimating the value of more data.
Match each learning-curve element with its role when estimating the benefit of more labeled data.
Order the reasoning steps for judging whether more data is likely to reach a target performance level.
Using a Validation Curve to Judge the Value of More Data
If the validation curve reaches the target after the data set doubles, what should be concluded?
Why Add the Target Line to a Learning Curve?
Which conclusion best matches the example learning curve?
A rough learning-curve estimate can support a tentative prediction about the value of adding more training data, but it does not guarantee the outcome.