# how to prove that $\hat \sigma^2$ is a consistent for $\sigma^2$

Consider a regression model $Y_n=X_n\beta +\varepsilon$, where $X_n$ is a $n \times p_n$ matrix, and $\varepsilon=(\varepsilon_1,...,\varepsilon_n)'$ consists of independent and identically distributed variables with $E(\varepsilon_1)=0$ and $Var(\varepsilon_1)=\sigma^2$. Suppose $\hat \sigma ^2$ is the estimator of $\sigma^2$. Let

$$\hat \sigma ^2=\frac{\left\|Y_n-X_n\hat\beta\right\|^2}{n-p_n}.$$

How to prove that $\hat \sigma^2$ consistent for $\sigma^2$?

• Can you assume normality of the error term $\epsilon$? Nov 16, 2016 at 10:21
• No. I cannot assume normality of $\varepsilon$ Nov 16, 2016 at 23:07
• If I assume normality of the error term, how can I use the assumption to prove that $\hat \sigma^2$ is consistent for $\sigma^2$? Nov 28, 2016 at 20:22
• I've edited the answer Dec 1, 2016 at 12:31

If you cannot assume normality then $\hat{e}_i = y_i - x_i'\hat{\beta} = e_i+x_i'\beta - x_i'\hat{\beta}=e_i+x_i'(\beta - \hat{\beta})$, recall that $\hat{\beta}_n \xrightarrow{p}\beta$, thus \begin{align} \frac{1}{n-p}||Y-X\hat{\beta}||^2 &= \frac{1}{n-p}\sum_{i=1}^n\hat{e}_i^2\\ &= \frac{1}{n-p}(\sum_{i=1}^n e^2_i + 2\sum_{i=1}^n e_ix_i'(\beta-\hat{\beta}) + (\beta-\hat{\beta})'\sum_{i=1}^n (x_ix_i')(\beta-\hat{\beta})), \end{align} where the second and the third terms converge in probability to $0$ and the first term goes to $\mathbb E{e_i^2}=\sigma^2$.

[1] Such a proof can be found in Econometrics by Bruce E. Hansen. If you can assume normality of the error term, then you can use the $\chi ^2$ distribution to prove consistency which can be slightly easier.

If you can assume normality then the proof becomes much easier as $$\mathbb{E}\hat{\sigma}^2 = \sigma^2,$$ then the MSE of $\hat{\sigma}^2$ equals its variance, i.e., \begin{align} \lim_{n\to \infty} MSE(\hat{\sigma}^2) &= \lim_{n\to \infty} Var(\frac{\sigma^2}{n-p} \frac{||Y-X\hat{\beta}||^2}{\sigma^2})\\ & =\lim_{n\to \infty}\frac{\sigma^4}{(n-p)^2} Var( \frac{||Y-X\hat{\beta}||^2}{\sigma^2})\\ & = \lim_{n\to \infty}\frac{2(n-p)\sigma^4}{(n-p)^2}\\ &= \lim_{n\to \infty}\frac{2\sigma^4}{(n-p)}=0. \end{align}

• I cannot assume normality. However, I can prove $\hat \sigma^2$ is unbiased estimator for $\sigma^2$. In order to prove the consistency, I need to prove $\lim Var(\hat \sigma^2)=0$. I stuck here. Nov 18, 2016 at 21:05
• I cannot understand the equation. For the first step, why $\lim_{n \rightarrow \infty} MSE(\hat \sigma^2)=\lim_{n \rightarrow \infty} Var (\frac{\sigma^2}{n-p} \left \| Y_n-X_n\hat \beta \right \|^2)$ ?? where does $\sigma^2$ come from? Dec 1, 2016 at 19:00
• Sorry, forgot to add the $\sigma^2$ in the denominator too. Anyhow, it is just a "trick" to get the $\chi^2$ distribution and then I used the fact that a variance of $\chi^2_{(n)}$ equals $2n$.. For more details on the you can see this answer. Dec 1, 2016 at 22:01
• Thanks for the hint. But in your case, X is followed normal distribution. Does the result still work for my case? Cause in my case, X is not said to be follow normal distribution. Dec 7, 2016 at 17:53
• No. The chi distribution is a result of the normal assumption. for non normal noise, you can use the first proof. It does not assumes any parametric structure of $\epsilon$. Dec 7, 2016 at 17:58