Google

A classifier with estimated bias of 15% and variance of 15% has both high bias and high variance: it performs poorly on the training set and performs even worse on the dev set. In this case, overfitting and underfitting terminology is hard to apply because both are happening.

High Bias and High Variance Bias-Variance Example

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Which statement best describes a classifier with an estimated 15% bias and 15% variance?

True or False: The overfitting/underfitting terminology applies clearly when a classifier has both high bias and high variance.

A classifier with high bias performs _____ on the training set.

Match each term to its defining characteristic in Andrew Ng's high bias/high variance scenario.

Order the steps to determine whether a classifier has high bias, high variance, or both.

In Andrew Ng's high bias/high variance example in Machine Learning Yearning, what are the two estimated error values?

True or False: A classifier with both high bias and high variance is simultaneously overfitting and underfitting.

In Andrew Ng's example, both the estimated bias and estimated variance are _____.

Match each classifier performance pattern to its error diagnosis.

Order the reasoning steps Andrew Ng uses to conclude a classifier has both high bias and high variance.

Question:
Based on Andrew Ng's Machine Learning Yearning course, explain why standard overfitting and underfitting terminology is difficult to apply to a classifier with 15% estimated bias and 15% estimated variance. Detail its performance on both training and dev sets in your explanation.

Sample answer:
Standard overfitting and underfitting terminology is difficult to apply because a classifier with 15% estimated bias and 15% estimated variance is simultaneously overfitting and underfitting. It underfits because it performs poorly on the training set, indicating high bias. At the same time, it overfits because its performance on the dev set is even worse, showing high variance. Since both phenomena occur at the same time, neither term alone suffices to describe the classifier's state.

Key points:
- The classifier is simultaneously overfitting and underfitting.
- Poor performance on the training set indicates high bias.
- Even worse performance on the dev set indicates high variance.

Rubric:
The response must explain that the classifier is simultaneously overfitting and underfitting, detail that it performs poorly on the training set (indicating high bias), and explain that its performance is even worse on the dev set (indicating high variance).

Explain why standard overfitting and underfitting terminology is difficult to apply to a classifier with high bias and high variance.

Case context:
You train a classifier and find that it achieves a 15% error rate on the training set, which you estimate as the bias. When evaluated on the dev set, the error rate increases to 30%, which represents an estimated variance of 15%.

Question:
Based on Andrew Ng's concepts, diagnose the bias and variance of this classifier. Explain how it performs on each dataset and why the traditional labels of overfitting or underfitting are problematic here.

Sample answer:
The classifier has both high bias and high variance. It performs poorly on the training set (15% bias) and performs even worse on the dev set (15% variance, 30% total error). In this scenario, applying either 'overfitting' or 'underfitting' terminology is problematic because both issues are happening simultaneously.

Key points:
- Identify the classifier as having both high bias and high variance.
- Explain poor training set performance represents high bias and even worse dev set performance represents high variance.
- State that overfitting and underfitting are occurring simultaneously, making standard labels problematic.

Rubric:
Evaluates whether the student correctly identifies the classifier as having both high bias and high variance, links these to poor training performance and even worse dev set performance, and states that overfitting and underfitting are occurring simultaneously.

Diagnosing a Classifier with 15% Training Error and 30% Dev Error

Question:
According to Andrew Ng's Machine Learning Yearning, how does a classifier with both high bias and high variance perform on the training set compared to the dev set?

Sample answer:
A classifier with both high bias and high variance performs poorly on the training set and performs even worse on the dev set.

Key points:
- Performs poorly on the training set.
- Performs even worse on the dev set.

Rubric:
The answer should state that the classifier performs poorly on the training set and performs even worse on the dev set.

Learn Before

Related