University of Michigan - Ann Arbor

Normalization is one of the techniques used to reduce the bias and variance in statistical learning. It helps to have all the variables centered around 0 and have the same variance.
We subtract all the values of each variable from their mean and divide them by their variance.
$$\mu = \frac{1}{m} \sum_{i = 1}^{m} X^{(i)}$$
$$\sigma^{2} = \frac{1}{m} \sum_{i = 1}^{m} (X^{(i)} - \mu)^2$$
$$X_{New} = \frac{X - \mu}{\sigma}$$

Normalization of Data

https://www.coursera.org/learn/deep-neural-network?specialization=deep-learning

Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

Normalization of training data may help reducing the bias and variance, and speeds up Gradient Descent, and consequently, training of the deep learning model, as shown in the figure.

Normalization Helps Deep Learning

Why do we normalize the inputs x?

The learning process of decision tree is essentially the process of selecting appropriate features, splitting and constructing tree nodes; The criterion of splitting node is determined by the information gain, information gain ratio and Gini coefficient before and after tree construction. These indicators are not related to the size of the current eigenvalue itself.

Learn Before

Related