# Show that the variance estimator of a linear regression is unbiased

I have to show that the variance estimator of a linear regression is unbiased, or simply $E\left[\widehat{\sigma}^2\right]=\sigma^2$. I have seen a few similar questions on here, but I think they are different enough to not answer my question.

I am given that $\widehat{\sigma}^2 = (n-2)^{-1} \sum_{i=1}^n (Y_i-(\alpha_0+\beta_0X_i))^2$.

Here are the steps I took: $$E\left[\widehat{\sigma}^2\right] = E\left[(n-2)^{-1} \sum_{i=1}^n (Y_i-(\alpha_0 + \beta_0 X_i)) \right] = (n-2)^{-1} E \left[ \sum_{i=1}^n (Y_i-(\alpha_0+\beta_0X_i))\right] = (n-2)^{-1} E\left[ \sum_{i=1}^n (Y_i-(\bar{Y}-\widehat{\beta}_0 \bar{X}_i) + \beta_0 X_i) \right]$$

Simplifying this by expanding and removing $0$ terms, I get $$E[\widehat{\sigma}^2]=(n-2)^{-1} \left(E\left[\sum_{i=1}^n (Y_i-\bar{Y})^2 \right] - E\left[ \widehat{\beta}\sum_{i=1}^n(X_i-\bar{X})^2\right]\right)$$

Then, for the first term, I plug in $Y_i=\alpha_0+\beta_0X_i+\epsilon_i$ and $\bar{Y}=\alpha_0+\beta_0 \bar{X}+\bar{\epsilon}$. For the second term, I set it equal to $E\left[\widehat{\beta}^2\right]$. (I know that so far my process is right as hinted by my professor)

I get:

$$E\left[\widehat{\sigma}^2\right]$$ $$= (n-2)^{-1} \left( E \left[\sum_{i=1}^n\beta_0^2(\bar{X}-X_i)^2 - 2\beta_0(\epsilon_i(\bar{X}-X_i)-\bar{\epsilon}(\bar{X}-X_i))+(\epsilon_i - \bar{\epsilon}^2 \right]-E\left[\widehat{\beta}^2\right]\right)$$

Since $\sum_{i=1}^n(\bar{X}-X_i)=0$, this reduces to

$$E\left[\widehat{\sigma}^2\right] = (n-2)^{-1} \left(E\left[\sum_{i=1}^n\beta_0^2(\bar{X}-X_i)^2\right] + E\left[\sum_{i=1}^n (\epsilon_i-\bar{\epsilon})^2\right]-E \left[ \widehat{\beta}^2 \right] \right).$$

However this is were I get stuck. I know $\hat{\beta}=\dfrac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$

• Is no matrix algebra relied upon here? I can quickly explain this using matrix algebra, but to do it without that would take some work. – Michael Hardy Nov 13 '17 at 6:20
• Unfortunately no matrix algebra :/ – Silvia Rossi Nov 13 '17 at 6:23

Start from showing that $$\frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{\sigma^2} \sim \chi^2_{(n-2)},$$ and then you'll have immediately the wanted result $$\mathbb{E}\left[ \frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{n-2}\right] =\sigma^2\frac{n-2}{n-2} = \sigma^2.$$