Reducing or eliminating regularization, such as L2 regularization, L1 regularization, or dropout, can reduce avoidable bias, but it increases variance.

Google

When a learning algorithm suffers from high avoidable bias, useful techniques include increasing model size, modifying input features based on error-analysis insights, reducing or eliminating regularization, and modifying the model architecture to better suit the problem. Adding more training data is not a helpful high-bias technique because it helps variance but usually has no significant effect on bias.

Techniques for Reducing Avoidable Bias

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Error-analysis insights can motivate creating additional input features that help eliminate a particular category of errors. These new features can help with both bias and variance; if adding features increases variance in practice, regularization will usually eliminate the increase.

Learn Before

Related