- **High Avoidable bias**: if it does not fit training set well on cost function (e.g., comparable to human level performance), you might train a bigger network, train longer, switch to a better optimization algorithm, like the Adam, or change the neural network architecture/hyperparameter search.
- **High Variance**: if it does not fit dev set on cost function, you may use regularization, increase the size of training set, or change the neural network architecture/hyperparameter search.
- If it does not fit test set well on cost function, you may change or increase the size of dev set.
- If it does not perform well in real world (e.g., happy users), you may change or increase the size of test set or change the cost function.

Troubleshooting a deep learning model

You will face mismatch problem when your training set comes from a different distribution than your dev and test sets, e.g., training set from webpage images and dev/test sets from mobile app uploaded images.

Data Mismatch Problem in Deep Learning

1. the difference between human error and training error: avoidable bias

2. the difference between training error and dev error: variations

Avoidable bias

- Human error VS training set error: avoidable bias
- Training set error VS training-dev set error: variance
- Training-dev set VS dev error: data mismatch
- Dev error VS test error: degree of overfitting to dev set

University of Michigan - Ann Arbor

For a supervised learning algorithm to do well, you need to check the following four assumptions:
- **Low Bias**: Fits training set well on cost function (e.g., comparable to human level performance)
- **Low Variance**: Fits dev set on cost function
- Fits test set well on cost function
- Performs well in real world (e.g., happy users)

Learn Before

Related

Learn After