Bias is a learner’s tendency to consistently learn the same wrong thing.
Variance is the tendency to learn random things irrespective of the real signal. Below, Figure 1 illustrates this with an analogy of throwing darts at a board.

University of Michigan - Ann Arbor

One reason that people manipulate hyperparameters is to improve the bias and variance that a given model contains. Bias is the difference between the average prediction of our model and the correct value which we are trying to predict. Models with high bias pay little attention to the training data and oversimplify the model, which will lead to high errors on training and test sets. Variance is the variability of model prediction for a given data point or a value which tells us spread of our data. High Variance models do not generalize based on previous data much, if at all, and thus while it may perform well on training data it will not on test sets. Briefly, the Bias vs Variance scale is a scale between over- and under-fitting of a model. By manipulating various hyperparameters, you can change between over- and under-fitting to get the best predictions

Bias and Variance in Deep Learning

Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55(10), 78-87. doi:10.1145/2347736.2347755

A few useful things to know about machine learning

Analogous Representation of Bias and Variance

Normalization of training data may help reducing the bias and variance, and speeds up Gradient Descent, and consequently, training of the deep learning model, as shown in the figure.

Learn Before

Related