Decide whether more data alone can close a large training gap.
Scenario: A fraud-detection model has a training error of 18%, which is still worse than the team's target of 8%. Its development error is 27%, so the dev set performs even worse than the training set. The team suggests that the only action needed is to collect more training examples.
Question: Is that plan enough to reach the target error? Explain using the relationship between training error, development error, and more data.
Sample answer: No. In this situation, collecting more data by itself is not enough. If the model already has training error above the target, more examples do not fix the fact that the model is underperforming on the training set. With the stated learning-curve pattern, additional data tends to reduce variance, but it does not directly solve the high-bias problem reflected by the high training error. The development error is also above the training error, so more data alone is not a complete remedy.
Key points:
- Training error is already higher than the target.
- More data does not by itself drive training error down to a much lower level.
- Dev error exceeds training error.
- The proposal is therefore insufficient on its own.
Rubric: The response should say that more data alone is not sufficient and should justify the answer by referring to the high training error and the dev-versus-training error gap.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Why More Data May Not Fix a High-Training-Error Model
Adding more training examples by itself can guarantee that dev error will reach a target level even when training error is still above that target.
When the training set gets larger, the training error can only stay the same or _____.
Match each learning-curve element to its role in diagnosing when extra data will not solve the problem.
Order the logic that shows more data by itself will not achieve the target performance.
Why more data alone cannot solve high training error
Decide whether more data alone can close a large training gap.
Why can’t adding more data by itself solve this learning-curve problem?
Which learning-curve pattern most clearly shows that collecting more data by itself will not solve the problem?
If training error does not improve as more labeled examples are added, development error can still be expected to drop below a lower target level from extra data alone.