University of Michigan - Ann Arbor

The left plot shows fitting a straight line to the data. This is not a very good fit. So, it is underfitting the data and we call it high bias. On the opposite end, if you fit an incredibly complex classifier, maybe you can fit the data perfectly, but that doesn't look like a great fit either. So there's a classifier of high variance and this is overfitting the data. And there might be some classifier in between, with a medium level of complexity, that maybe fits it correctly.

Overfitting/Underfitting vs. Bias/Variance in Supervised Machine Learning

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

For a supervised learning algorithm to do well, you need to check the following four assumptions:
- **Low Bias**: Fits training set well on cost function (e.g., comparable to human level performance)
- **Low Variance**: Fits dev set on cost function
- Fits test set well on cost function
- Performs well in real world (e.g., happy users)

Chain of Assumptions in Supervised Statistical Learning

Fortmann-Roe, S. (n.d.). (2012). Understanding the Bias-Variance Tradeoff. Retrieved from
[scott.fortmann-roe.com/docs/BiasVariance.html](https://scott.fortmann-roe.com/docs/BiasVariance.html) 

Understanding the Bias-Variance Tradeoff

Training and test error behave diﬀerently. At the left end of the graph, training error and test error are both high. This is the underﬁtting regime. As we increase capacity, training error decreases, but the gap between training and generalization error increases. Eventually, the size of this gap outweighs the decrease in training error, and we enter the overﬁtting regime, where capacity is too large, above the optimal capacity.

Learn Before

Related