Diagnosing a Classifier with 15% Training Error and 30% Dev Error
Case context: You train a classifier and find that it achieves a 15% error rate on the training set, which you estimate as the bias. When evaluated on the dev set, the error rate increases to 30%, which represents an estimated variance of 15%.
Question: Based on Andrew Ng's concepts, diagnose the bias and variance of this classifier. Explain how it performs on each dataset and why the traditional labels of overfitting or underfitting are problematic here.
Sample answer: The classifier has both high bias and high variance. It performs poorly on the training set (15% bias) and performs even worse on the dev set (15% variance, 30% total error). In this scenario, applying either 'overfitting' or 'underfitting' terminology is problematic because both issues are happening simultaneously.
Key points:
- Identify the classifier as having both high bias and high variance.
- Explain poor training set performance represents high bias and even worse dev set performance represents high variance.
- State that overfitting and underfitting are occurring simultaneously, making standard labels problematic.
Rubric: Evaluates whether the student correctly identifies the classifier as having both high bias and high variance, links these to poor training performance and even worse dev set performance, and states that overfitting and underfitting are occurring simultaneously.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Which statement best describes a classifier with an estimated 15% bias and 15% variance?
True or False: The overfitting/underfitting terminology applies clearly when a classifier has both high bias and high variance.
A classifier with high bias performs _____ on the training set.
Match each term to its defining characteristic in Andrew Ng's high bias/high variance scenario.
Order the steps to determine whether a classifier has high bias, high variance, or both.
In Andrew Ng's high bias/high variance example in Machine Learning Yearning, what are the two estimated error values?
True or False: A classifier with both high bias and high variance is simultaneously overfitting and underfitting.
In Andrew Ng's example, both the estimated bias and estimated variance are _____.
Match each classifier performance pattern to its error diagnosis.
Order the reasoning steps Andrew Ng uses to conclude a classifier has both high bias and high variance.
Explain why standard overfitting and underfitting terminology is difficult to apply to a classifier with high bias and high variance.
Diagnosing a Classifier with 15% Training Error and 30% Dev Error
How does a classifier with high bias and high variance perform on the training and dev sets?