For a sample of size $m$ from a Normally distributed random variable with paramters $\mu$ and $\sigma$, the sample variance, which is the sum of the squared deviations from the mean over the number of samples, the estimator
	$$\mathbb{E}[\hat{\sigma'}^2_m] = \mathbb{E}\left[\frac{1}{m}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right]$$
Is a biased estimator for the variance $\sigma^2$. 

To see why, consider that the sample variance, 
	$$\mathbb{E}[\hat{\sigma}^2_m] =\mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right]$$
is an unbiased estimator for the variance $\sigma^2$. Then 
$\textrm{bias}(\hat{\sigma}_m^2 = \mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right] - \sigma^2 = 0$, so $ \mathbb{E}(\hat{\sigma}_m^2) = \mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right] = \sigma^2 \not =  \mathbb{E}\left[\frac{1}{m}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right] = \mathbb{E}(\hat{\sigma'}_m^2)$, since $m-1 \not = m$. 


Example of Biased Estimators

See this [youtube video](https://www.youtube.com/watch?v=D1hgiAla3KI) for a explanation for why 
$$\mathbb{E}[\hat{\sigma}^2_m] =\mathbb{E}\left[\frac{1}{m - 1}\sum_{i=1}^m \left(x^{(i)} - \hat{\mu}_m\right)^2\right]$$
is an unbiased estimator for the variance $\sigma^2$. 

Johns Hopkins University

When we take $n$ $i.i.d.$ samples from a Bernoulli distribution with parameter $\theta$, the mean of those samples is an unbiased estimator for $\theta$

Similarly, the mean of $n$ $i.i.d.$ samples from a Normal distribution is an unbiased estimator for the $\mu$ paramter of the Normal distirbution. 

Examples of Unbiased Estimators

When we have $m$ $i.i.d$ samples  $\{x^{(1)},...,x^{(m)}\}$ from a Bernoulli distribution with parameter $\theta$, such that: 
	$$P(x^{(i)},\theta) = \theta^{x^{(i)}} (1-\theta)^{(1-x^{i})}$$,
a unbiased estimator for $\theta$ is the mean of those samples:
	$\hat{\theta}_m = \frac{1}{m} \sum_{i=1}^m x^{(i)}$


Unbiased Estimator for Bernoulli Parameter

Unbiased Estimator for Normal $\sigma^2$ parameter

When we have $m$ $i.i.d$ samples  $\{x^{(1)},...,x^{(m)}\}$ from a Normal distribution with parameter $\mu$ and $\sigma$, such that: 
	$$P(x^{(i)};\mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}}e^{\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$$,
a unbiased estimator for $\mu$ is the mean of those samples:
	$\hat{\mu}_m = \frac{1}{m} \sum_{i=1}^m x^{(i)}$


Learn Before

Related

Learn After