2
$\begingroup$

Suppose the variance of an iid sequence of random variables is formed according to

$$\widehat \sigma = \frac{1}{n} \sum^{n}_{m=1}(X_m - \widehat \mu )^2$$

$\widehat \mu$ is the sample mean. Find the expected value of this estimator and show that it is biased

I don't know how to calculate the expected value from this kind of variance,can anyone teach me?

$\endgroup$
  • 1
    $\begingroup$ I would say the sample variance is defined that way, rather than simply that the variance is defined that way, and also write $\widehat\sigma^2$ rather than $\widehat\sigma. \qquad$ $\endgroup$ – Michael Hardy May 8 '18 at 2:46
3
$\begingroup$

We have to show that $\mathbb E(s^2)\neq \sigma^2$. I write down the full calculation with some explanations. If there comes a step where you think you can go on by yourself just don´t continue reading.

$\mathbb E(s^2)=\mathbb E\left[\frac{1}{n}\sum_{i=1}^n (X_i-\overline X )^2\right]$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\overline X)^2 \right] \quad | \pm \mu$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n \left[(X_i-\mu)-(\overline X-\mu) \right]^2 \right] \quad$

multipliying out

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n \left[(X_i-\mu)^2-2(\overline X-\mu)(X_i-\mu)+(\overline X-\mu)^2 \right]\right] \quad$

writing for each summand a sigma sign

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-2(\overline X-\mu)\sum_{i=1}^n(X_i-\mu)+\sum_{i=1}^n(\overline X-\mu)^2 \right] \quad$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-2(\overline X-\mu)\color{red}{\sum_{i=1}^n(X_i-\mu)}+n(\overline X-\mu)^2 \right] \quad$


transforming the red term

$\sum_{i=1}^n(X_i-\mu)=n\cdot \overline X-n\cdot \mu$

Thus $2(\overline X-\mu)\color{red}{\sum_{i=1}^n(X_i-\mu)}=2(\overline X-\mu)\cdot (n\cdot \overline X-n\cdot \mu)=2n( \overline X- \mu)^2$


$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-2n( \overline X- \mu)^2+n(\overline X-\mu)^2 \right] \quad$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-n( \overline X- \mu)^2\right] \quad$

$=\frac{1}{n}\left[\sum_{i=1}^n \mathbb E\left[(X_i-\mu)^2\right]-nE\left[( \overline X- \mu)^2\right]\right] \quad$

We know, that $\mathbb E\left[(X_i-\mu)^2\right]=\sigma^2$ and $E\left[( \overline X- \mu)^2\right]=\sigma_{\overline x}^2=\frac{\sigma^2}{n}$ Thus we get

$=\frac{1}{n}\left[n \cdot \sigma ^2-n \frac{\sigma ^2}{n}\right]=\frac{1}{n} \sigma^2 \cdot (n-1)$

$\endgroup$
1
$\begingroup$

Showing simply that it is biased is easier than finding its expected value: $$ (X_m - \mu )^2 = \big( (X_m - \widehat\mu) + (\widehat\mu - \mu) \big)^2 = \underbrace{ (X_m-\widehat\mu)^2}_A + \underbrace{2(X_m-\widehat\mu)(\widehat\mu-\mu)}_B + \underbrace{(\widehat\mu-\mu)^2}_C $$ The expected value of the sum of the terms labeled $B$ is $0,$ because the sum of $X_m-\widehat\mu$ is $0$ (later note: This does not mean that $X_m-\widehat\mu)$ is $0;$ but only that their sum is $0$), and the other factor, $2(\widehat\mu-\mu),$ does not change as $m$ goes from $1$ to $n.$

Therefore the sum of the terms labeled $A,B,C$ is the sum of those labeled $A$ and those labeled $C.$

Therefore the expected value of the sum of the terms on the left is bigger than the expected value of the sum of the terms labeled $A.$

And the expected value of the sum of the terms on the left is $n\sigma^2.$

$\endgroup$
  • $\begingroup$ why is $X_m $equal to $\hat μ $?if the $X_m $ is equal to $\hat μ $,the the sum of $A ,B,C$ should be the sum of $C$.What is relation between $\hat μ $ and $ μ $? $\endgroup$ – XM551 May 8 '18 at 6:07
  • $\begingroup$ if the $X_m $ is equal to $\hat μ $,the the sum of $A ,B,C$ should be the sum of $C$ $\endgroup$ – XM551 May 8 '18 at 6:09
  • $\begingroup$ @XM551 : $X_m$ is not equal to $\widehat\mu.$ Rather, $\sum_m X_m$ is equal to $\sum_m \widehat \mu. \qquad$ $\endgroup$ – Michael Hardy May 8 '18 at 19:06
  • $\begingroup$ @XM551 : I said THE SUM OF $X_m-\widehat\mu$ is equal to $0,$ NOT that $X_m-\widehat\mu$ is itself equal to $0. \qquad$ $\endgroup$ – Michael Hardy May 8 '18 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.