Google

When training-set performance is already close to the optimal error rate, there is little room for bias improvement. If the algorithm is not generalizing well to the dev set, there is ample room for variance improvement.

Learning Curve Pattern for High Variance

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Identifying High Variance from Learning Curves

Room for Bias Improvement

Adding _____ to Close the Error Gap

Matching Learning Curve Concepts

Analyzing a High Variance Learning Curve

Question:
Describe a scenario where an algorithm achieves a training error close to the optimal error rate but has a much higher dev error. Explain what this implies about bias and variance, and discuss the potential for improvement in both areas.

Sample answer:
In this scenario, the algorithm does well on the training set, meaning the training error is close to the optimal error rate. This indicates that the bias is small. However, because the dev error is much higher, the algorithm is not generalizing well to previously unseen data, which means the variance is large. Consequently, there is little room for bias improvement, but there is ample room for variance improvement. Adding more training data is a strategy that would probably help close the gap.

Key points:
- Training error near the optimal rate indicates small bias.
- High dev error relative to training error indicates large variance.
- There is little room for bias improvement.
- There is ample room for variance improvement.
- Adding training data can help close the generalization gap.

Rubric:
A strong response will correctly identify low bias and high variance based on the described errors, and explain that improvement efforts should focus on variance (e.g., adding more data) rather than bias.

Assessing Error Improvement Potential

Case context:
You are reviewing a learning curve for a model where the optimal error rate is known to be around 14%. Your algorithm's training error is currently at 14.5%, but its dev error is at 26%.

Question:
Based on this learning curve, what should you diagnose about the model's bias and variance, and where is the room for improvement?

Sample answer:
The model's training error of 14.5% is very close to the optimal error rate of 14%, indicating that the bias is small and there is not much room for improvement in terms of bias. However, the dev error of 26% is much higher than the training error, showing the model is not generalizing well. This indicates high variance, meaning there is ample room for improvement in errors due to variance, potentially by adding more training data.

Key points:
- Training error near optimal implies small bias.
- A large gap between dev error and training error implies large variance.
- There is little room to improve bias.
- There is ample room to improve variance.

Rubric:
The learner must diagnose low bias with little room for bias improvement, and high variance with ample room for variance improvement.

Diagnosing a Dev Error Gap

Question:
If an algorithm's training error is close to the optimal error rate, why is there more room for variance improvement than bias improvement?

Sample answer:
Because the training performance is already near the optimal rate, the bias is small, leaving little room for bias improvement. If the algorithm fails to generalize, the dev error will be much higher, meaning the variance is large and there is ample room to reduce it.

Key points:
- Training error near the optimal rate means small bias.
- Poor generalization means large variance.
- Ample room exists for variance improvement.

Rubric:
The answer must state that low training error leaves little room for bias reduction, while poor generalization leaves ample room to reduce variance.

Learn Before

Related