What Does a Large Gap Between Training Error and Best Possible Error Suggest?
Question: A classifier has 15% training error, while the best attainable error is essentially 0%. What does that imply about the likely usefulness of changes aimed at reducing bias?
Sample answer: It suggests that bias-reducing changes could help a lot, because there is still a large gap between the current training performance and the best achievable performance.
Key points:
- There is substantial room to improve performance.
- Bias-reducing changes are likely to be useful.
Rubric: The answer should state that bias-reducing changes are likely to be fruitful because the model still has a large amount of room for improvement.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Judging Avoidable Bias from the Training Set
A model with 8% training error and 7% best achievable error shows strong evidence of high bias.
Room to Improve When the Best Possible Error Is Near Zero
Matching Bias Clues to Error Gaps
Diagnosing Bias by Comparing Training Error to a Practical Baseline
Why the Same Training Error Can Mean Different Things
Estimating Bias From Training Error and a Human Performance Baseline
What Does a Large Gap Between Training Error and Best Possible Error Suggest?
How to judge whether there is still room to reduce avoidable bias
Can training error by itself tell you whether a bias-reducing change is worthwhile?