An estimator $\hat{\theta}_m$ is called biased if $$\textrm{bias}(\hat{\theta}_m) \not = 0$$. 

By the definition of bias, this implies $\mathbb{E}(\hat{\theta}_m) \not = \theta$. In other words, the expected value of the estimator $\hat{\theta}_m$ is not equal to the actual value of the paramter used to generate the data, $\theta$. 

Biased Estimators

When we take $$n$$ i.i.d. samples from a Bernoulli distribution with parameter $$\theta$$, the mean of those samples is an unbiased estimator for $$\theta$$. Similarly, the mean of $$n$$ i.i.d. samples from a Normal distribution is an unbiased estimator for the $$\mu$$ parameter of the Normal distribution.

Examples of Unbiased Estimators

An estimator $$\hat{\theta}_m$$ is called unbiased if $$\textrm{bias}(\hat{\theta}_m) = 0$$. By the definition of bias, $$\mathrm{bias}(\hat \theta_m) = \mathbb{E}(\hat \theta_m) - \theta$$, this implies $$\mathbb{E}(\hat{\theta}_m) = \theta$$. In other words, when an estimator is unbiased, the expected value of the estimator $$\hat{\theta}_m$$ is equal to the actual value of the parameter used to generate the data, $$\theta$$.

Johns Hopkins University

Google

$$\mathrm{bias}(\hat{\theta}_m) = \mathbb{E}(\hat{\theta}_m) - \theta$$, where $$\hat{\theta}_m$$ is the estimator of a parameter, $$\theta$$ is the true value of that parameter, and $$m$$ is the sample size/number of data points. The bias of an estimator for a parameter is the difference between the expected value that the estimator will take when trained on data and the actual value of the parameter used to generate the data.

Mathematical Definition of Estimator Bias

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related

Learn After