# Prove that the the variance estimator $\widehat{\sigma}^2=MSE/(n-2)$ is biased is the simple linear regression model

This is in scope of the simple linear model. Im trying to prove that $\mathbb{E}\left(\widehat{\sigma}^2\right) = \sigma^2$ for

$$\widehat{\sigma}^2 = \frac{1}{n-2}\sum^n_{i=1} \left(y_i-\widehat{y}_i\right)^2$$

where $$Y_i\sim N(\beta_0+\beta_1x_i,\sigma^2)$$ and $\widehat{y_i},i=1,2,3,...,n$ are pedricted values and $y_1,y_2,...,y_n$ is a sample from $Y_i$

You have a normally distributed random vector in an $(n-2)$-dimensional Euclidean space. Its expected value is $0$ and the normal distribution is spherically symmetric. Hence the square of its norm is distributed as $\sigma^2\chi^2_{n-2}$, so the expected value of the square of the norm is $\sigma^2(n-2)$.
Indeed, if you project $Y$ onto that $(n-2)$-dimensional space, getting $\hat\varepsilon$, and then let $U_1,\ldots, U_{n-2}$ be the coordinates of $\hat\varepsilon$ with respect to an orthonormal basis of that space then you have $$U_1,\ldots,U_{n-2}\sim\mathrm{i.i.d.}\ N(0,\sigma^2)$$ and $$\varepsilon_1^2+\cdots+\varepsilon_n^2=U_1^2+\cdots+U_{n-2}^2.$$
• What would $H$ be in the simple case ? i.e n = 1 ?? – Teodor Fredriksson Sep 12 '14 at 21:18
• In case $n=1$, $H$ is just a $1\times1$ matrix whose only entry is $1$. You don't do regression with only one data point. And with only two data points with different $x$ values, the straight-line fit is perfect. Let's try $n=3$. You have three $x$ values: $1$, $2$, and $3$. Then $X=\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}$, and $H=X(X^TX)^{-1}X^T= (1/6) \begin{bmatrix} 5 & 2 & -1 \\ 2 & 2 & 2 \\ -1 & 2 & 5 \end{bmatrix}$. This is a $3\times3$ symmetric idempotent matrix of rank $2$. ${}\qquad{}$ – Michael Hardy Sep 12 '14 at 22:44
• Call the $y$ values $y_1,y_2,y_3$. Then $HY=\begin{bmatrix} (5y_1+2y_2-y_3)/6 \\ (y_1+y_2+y_3)/3 \\ (-y_1+2y_2+5y_3)/6 \end{bmatrix}$. That is the vector $\widehat Y$ of fitted values. And $Y-\widehat Y$ is the vector $\widehat\varepsilon$ of residuals. ${}\qquad{}$ – Michael Hardy Sep 12 '14 at 22:46
• . . . . and may I add that if $n=1$ then the $n-2$ by which you divide is negative, and if $n=2$, then you'd be dividing by $0$. But if $n=2$ then the numerator would also be $0$, i.e. the sum of squares of residuals would be $0$ because the fit would be perfect. A jocular rule of experimentation is: if a straight-line fit is required, then obtain only two data points. ${}\qquad{}$ – Michael Hardy Sep 12 '14 at 22:57