1
$\begingroup$

I have to show that the variance estimator of a linear regression is unbiased, or simply $E\left[\widehat{\sigma}^2\right]=\sigma^2$. I have seen a few similar questions on here, but I think they are different enough to not answer my question.

I am given that $\widehat{\sigma}^2 = (n-2)^{-1} \sum_{i=1}^n (Y_i-(\alpha_0+\beta_0X_i))^2$.

Here are the steps I took: $$E\left[\widehat{\sigma}^2\right] = E\left[(n-2)^{-1} \sum_{i=1}^n (Y_i-(\alpha_0 + \beta_0 X_i)) \right] = (n-2)^{-1} E \left[ \sum_{i=1}^n (Y_i-(\alpha_0+\beta_0X_i))\right] = (n-2)^{-1} E\left[ \sum_{i=1}^n (Y_i-(\bar{Y}-\widehat{\beta}_0 \bar{X}_i) + \beta_0 X_i) \right]$$

Simplifying this by expanding and removing $0$ terms, I get $$E[\widehat{\sigma}^2]=(n-2)^{-1} \left(E\left[\sum_{i=1}^n (Y_i-\bar{Y})^2 \right] - E\left[ \widehat{\beta}\sum_{i=1}^n(X_i-\bar{X})^2\right]\right)$$

Then, for the first term, I plug in $Y_i=\alpha_0+\beta_0X_i+\epsilon_i$ and $\bar{Y}=\alpha_0+\beta_0 \bar{X}+\bar{\epsilon}$. For the second term, I set it equal to $E\left[\widehat{\beta}^2\right]$. (I know that so far my process is right as hinted by my professor)

I get:

$$E\left[\widehat{\sigma}^2\right]$$ $$ = (n-2)^{-1} \left( E \left[\sum_{i=1}^n\beta_0^2(\bar{X}-X_i)^2 - 2\beta_0(\epsilon_i(\bar{X}-X_i)-\bar{\epsilon}(\bar{X}-X_i))+(\epsilon_i - \bar{\epsilon}^2 \right]-E\left[\widehat{\beta}^2\right]\right)$$

Since $\sum_{i=1}^n(\bar{X}-X_i)=0$, this reduces to

$$E\left[\widehat{\sigma}^2\right] = (n-2)^{-1} \left(E\left[\sum_{i=1}^n\beta_0^2(\bar{X}-X_i)^2\right] + E\left[\sum_{i=1}^n (\epsilon_i-\bar{\epsilon})^2\right]-E \left[ \widehat{\beta}^2 \right] \right).$$

However this is were I get stuck. I know $\hat{\beta}=\dfrac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$

$\endgroup$
  • $\begingroup$ Is no matrix algebra relied upon here? I can quickly explain this using matrix algebra, but to do it without that would take some work. $\endgroup$ – Michael Hardy Nov 13 '17 at 6:20
  • $\begingroup$ Unfortunately no matrix algebra :/ $\endgroup$ – Silvia Rossi Nov 13 '17 at 6:23
0
$\begingroup$

Start from showing that $$ \frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{\sigma^2} \sim \chi^2_{(n-2)}, $$ and then you'll have immediately the wanted result $$ \mathbb{E}\left[ \frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{n-2}\right] =\sigma^2\frac{n-2}{n-2} = \sigma^2. $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.