Learn Before
Concept
Batch Norm as Regularization in Deep Learning
Batch Norm has a slight regularization effect. Since we shift and scale z based on the mean and variance of only a single batch, it adds some noise into each hidden layer activation. So like dropout, it will add noise to each hidden layer’s activation. However, the effect is so small that we might want to use it together with other regularization methods like dropout. If one uses a larger mini-batch size, it'd create less noise and thereby reduce the regularization effect.
0
2
Updated 2021-03-31
Contributors are:
Who are from:
Tags
Data Science