For a sample of size $m$ from a Normally distributed random variable with paramters $\mu$ and $\sigma$, the sample variance, which is the sum of the squared deviations from the mean over the number of samples, the estimator
	$$\mathbb{E}[\hat{\sigma'}^2_m] = \mathbb{E}\left[\frac{1}{m}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right]$$
Is a biased estimator for the variance $\sigma^2$. 

To see why, consider that the sample variance, 
	$$\mathbb{E}[\hat{\sigma}^2_m] =\mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right]$$
is an unbiased estimator for the variance $\sigma^2$. Then 
$\textrm{bias}(\hat{\sigma}_m^2 = \mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right] - \sigma^2 = 0$, so $ \mathbb{E}(\hat{\sigma}_m^2) = \mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right] = \sigma^2 \not =  \mathbb{E}\left[\frac{1}{m}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right] = \mathbb{E}(\hat{\sigma'}_m^2)$, since $m-1 \not = m$. 


Example of Biased Estimators

An estimator $$\hat{\theta}_m$$ is called biased if $$\textrm{bias}(\hat{\theta}_m) \not = 0$$. By the definition of bias, this implies $$\mathbb{E}(\hat{\theta}_m) \not = \theta$$. In other words, the expected value of the estimator $$\hat{\theta}_m$$ is not equal to the actual value of the parameter used to generate the data, $$\theta$$.

Johns Hopkins University

Google

An estimator $$\hat{\theta}_m$$ is called unbiased if $$\textrm{bias}(\hat{\theta}_m) = 0$$. By the definition of bias, $$\mathrm{bias}(\hat \theta_m) = \mathbb{E}(\hat \theta_m) - \theta$$, this implies $$\mathbb{E}(\hat{\theta}_m) = \theta$$. In other words, when an estimator is unbiased, the expected value of the estimator $$\hat{\theta}_m$$ is equal to the actual value of the parameter used to generate the data, $$\theta$$.

Unbiased Estimator

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

When we take $$n$$ i.i.d. samples from a Bernoulli distribution with parameter $$\theta$$, the mean of those samples is an unbiased estimator for $$\theta$$. Similarly, the mean of $$n$$ i.i.d. samples from a Normal distribution is an unbiased estimator for the $$\mu$$ parameter of the Normal distribution.

Learn Before

Related

Learn After