Consider a regression model $Y_n=X_n\beta +\varepsilon$, where $X_n$ is a $n \times p_n$ matrix, and $\varepsilon=(\varepsilon_1,...,\varepsilon_n)'$ consists of independent and identically distributed variables with $E(\varepsilon_1)=0$ and $Var(\varepsilon_1)=\sigma^2$. Suppose $\hat \sigma ^2$ is the estimator of $\sigma^2$. Let

$$\hat \sigma ^2=\frac{\left\|Y_n-X_n\hat\beta\right\|^2}{n-p_n}.$$

How to prove that $\hat \sigma^2$ consistent for $\sigma^2$?

  • $\begingroup$ Can you assume normality of the error term $\epsilon$? $\endgroup$
    – V. Vancak
    Nov 16, 2016 at 10:21
  • $\begingroup$ No. I cannot assume normality of $\varepsilon$ $\endgroup$ Nov 16, 2016 at 23:07
  • $\begingroup$ If I assume normality of the error term, how can I use the assumption to prove that $\hat \sigma^2$ is consistent for $\sigma^2$? $\endgroup$ Nov 28, 2016 at 20:22
  • $\begingroup$ I've edited the answer $\endgroup$
    – V. Vancak
    Dec 1, 2016 at 12:31

1 Answer 1


If you cannot assume normality then $\hat{e}_i = y_i - x_i'\hat{\beta} = e_i+x_i'\beta - x_i'\hat{\beta}=e_i+x_i'(\beta - \hat{\beta})$, recall that $\hat{\beta}_n \xrightarrow{p}\beta$, thus \begin{align} \frac{1}{n-p}||Y-X\hat{\beta}||^2 &= \frac{1}{n-p}\sum_{i=1}^n\hat{e}_i^2\\ &= \frac{1}{n-p}(\sum_{i=1}^n e^2_i + 2\sum_{i=1}^n e_ix_i'(\beta-\hat{\beta}) + (\beta-\hat{\beta})'\sum_{i=1}^n (x_ix_i')(\beta-\hat{\beta})), \end{align} where the second and the third terms converge in probability to $0$ and the first term goes to $\mathbb E{e_i^2}=\sigma^2$.

[1] Such a proof can be found in Econometrics by Bruce E. Hansen. If you can assume normality of the error term, then you can use the $\chi ^2$ distribution to prove consistency which can be slightly easier.

If you can assume normality then the proof becomes much easier as $$ \mathbb{E}\hat{\sigma}^2 = \sigma^2, $$ then the MSE of $\hat{\sigma}^2$ equals its variance, i.e., \begin{align} \lim_{n\to \infty} MSE(\hat{\sigma}^2) &= \lim_{n\to \infty} Var(\frac{\sigma^2}{n-p} \frac{||Y-X\hat{\beta}||^2}{\sigma^2})\\ & =\lim_{n\to \infty}\frac{\sigma^4}{(n-p)^2} Var( \frac{||Y-X\hat{\beta}||^2}{\sigma^2})\\ & = \lim_{n\to \infty}\frac{2(n-p)\sigma^4}{(n-p)^2}\\ &= \lim_{n\to \infty}\frac{2\sigma^4}{(n-p)}=0. \end{align}

  • $\begingroup$ I cannot assume normality. However, I can prove $\hat \sigma^2$ is unbiased estimator for $\sigma^2$. In order to prove the consistency, I need to prove $\lim Var(\hat \sigma^2)=0$. I stuck here. $\endgroup$ Nov 18, 2016 at 21:05
  • $\begingroup$ I cannot understand the equation. For the first step, why $\lim_{n \rightarrow \infty} MSE(\hat \sigma^2)=\lim_{n \rightarrow \infty} Var (\frac{\sigma^2}{n-p} \left \| Y_n-X_n\hat \beta \right \|^2)$ ?? where does $\sigma^2$ come from? $\endgroup$ Dec 1, 2016 at 19:00
  • $\begingroup$ Sorry, forgot to add the $\sigma^2$ in the denominator too. Anyhow, it is just a "trick" to get the $\chi^2$ distribution and then I used the fact that a variance of $\chi^2_{(n)}$ equals $2n$.. For more details on the you can see this answer. $\endgroup$
    – V. Vancak
    Dec 1, 2016 at 22:01
  • $\begingroup$ Thanks for the hint. But in your case, X is followed normal distribution. Does the result still work for my case? Cause in my case, X is not said to be follow normal distribution. $\endgroup$ Dec 7, 2016 at 17:53
  • 1
    $\begingroup$ No. The chi distribution is a result of the normal assumption. for non normal noise, you can use the first proof. It does not assumes any parametric structure of $\epsilon$. $\endgroup$
    – V. Vancak
    Dec 7, 2016 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.