Google

A learning curve can indicate both significant bias and significant variance when training error is much higher than the desired performance level and dev error is much higher than training error. In this case, the algorithm needs changes that reduce both bias and variance.

Learning Curve Pattern for Both High Bias and High Variance

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Identifying Simultaneous High Bias and High Variance

Dev Error in High Bias and Variance Scenarios

Necessary Algorithmic Adjustments for _____ Problem

Interpreting Learning Curve Components

Steps for Diagnosing High Bias and Variance

Question:
Explain what a learning curve looks like when an algorithm has both high bias and high variance, and what general conclusion must be drawn about next steps for the algorithm.

Sample answer:
The learning curve will show a training error that is large, meaning it is much higher than the desired level of performance, which indicates high bias. Additionally, the dev error will be much larger than the training error, which indicates high variance. The conclusion drawn from this pattern is that you will have to find a way to reduce both bias and variance in your algorithm.

Key points:
- Training error is much higher than desired performance.
- Dev error is much higher than training error.
- Algorithm changes must aim to reduce both bias and variance.

Rubric:
A strong answer must clearly state the relationship between training error and desired performance (high bias), the relationship between dev error and training error (high variance), and the required action (reducing both).

Addressing Simultaneous Bias and Variance Issues

Case context:
You are reviewing the learning curve for a new prediction system. The desired error rate is 2%, but the training error has plateaued at 15%. Furthermore, the dev error is currently sitting at 28%.

Question:
Based on this learning curve data, what should you diagnose about the system's bias and variance, and what is your broad recommendation for the algorithm?

Sample answer:
The system has significant bias because the training error (15%) is much higher than the desired level of performance (2%). It also has significant variance because the dev error (28%) is much larger than the training error. The broad recommendation is to find ways to reduce both bias and variance in the algorithm.

Key points:
- Significant bias is present due to 15% training error vs 2% desired performance.
- Significant variance is present due to 28% dev error vs 15% training error.
- Recommendation is to reduce both bias and variance.

Rubric:
The response must correctly identify high bias due to the high training error, identify high variance due to the gap between training and dev error, and recommend reducing both.

Diagnostic Report on a New System

Question:
What specific relationship between dev error and training error indicates significant variance when an algorithm already shows significant bias?

Sample answer:
The dev error is much larger than the training error.

Key points:
- Dev error is larger than training error.

Rubric:
The answer should explicitly state that dev error is much higher or larger than the training error.

Learn Before

Related