Google

In the cat classification example, estimated bias of 1% and variance of 10% indicates high variance. The classifier has very low training error but fails to generalize to the dev set, which is also called overfitting.

High Variance Bias-Variance Example for Cat Classification

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

In the cat classification example, if training error is ~1% and dev error is 11%, what is the estimated variance?

A classifier with high variance has very low training error but fails to generalize to the dev set.

When a classifier has very low training error but high dev error, this is also called _____.

Match each metric or concept to its correct description in the cat classification bias-variance example.

Order the steps used in ML Yearning to diagnose high variance in the cat classification example.

Which pattern of training and dev set errors characterizes a high variance classifier in the cat example?

In ML Yearning's cat classification example, a classifier diagnosed with high variance has high training error.

In the cat classification example, variance is estimated as _____ minus training error.

Match each term to its role in diagnosing high variance from the cat classification example.

Order the reasoning steps to conclude the cat classifier is overfitting based on its error metrics.

Question:
Explain why a cat classifier with 1% estimated bias and 10% estimated variance is considered to have "high variance," and describe the relationship between its performance on the training set versus the development set.

Sample answer:
A classifier with 1% bias and 10% variance has "high variance" because the gap between its training error and development set error is large (10%). This indicates that while the classifier has very low training error (approx 1%), it is failing to generalize well to the dev set (11% error). This phenomenon, where the model performs exceptionally well on the training data but poorly on unseen data, is known as overfitting.

Key points:
- Variance is estimated at 10% (the gap between dev and training error).
- The classifier has very low training error, implying low bias (1%).
- The model fails to generalize to the dev set.
- This condition is referred to as overfitting.

Rubric:
The response should explicitly define the gap between training and dev error as the variance (10%), acknowledge the low training error indicating low bias (1%), and correctly identify the failure to generalize as overfitting.

Analyzing the Cat Classifier's Error Metrics

Case context:
You are developing a cat classification model. After training, your algorithm achieves a 1% error rate on your training set but an 11% error rate on your development set. An ideal human classifier achieves nearly 0% error.

Question:
Based on the error metrics provided, diagnose the primary problem with this classifier. What specific ML terminology describes this situation, and what do the 1% and 10% values represent?

Sample answer:
The primary problem is that the classifier is overfitting to the training data. The 1% training error represents the estimated bias, which is very low. The 10% difference between the dev set error (11%) and the training error (1%) represents the variance. Because the variance is high, the model is failing to generalize to new data.

Key points:
- Diagnose the problem as overfitting or failing to generalize.
- Identify the 1% metric as the estimated bias.
- Calculate or identify the 10% gap as the estimated variance.
- Recognize that high variance is the primary issue.

Rubric:
A correct diagnosis must identify the issue as overfitting or high variance. It should correctly assign the 1% error to estimated bias and calculate the 10% difference as estimated variance.

Diagnosing Overfitting in a Cat Image Recognizer

Question:
In the cat classification example, what term is used to describe a model that has very low training error but fails to generalize to the development set?

Sample answer:
This phenomenon is called overfitting.

Key points:
- Overfitting is the correct term.
- It corresponds to low training error and failing to generalize.

Rubric:
The learner must correctly provide the term "overfitting".

Learn Before

Related