1
$\begingroup$

Consider a regression model $Y_n=X_n\beta +\varepsilon$, where $X_n$ is a $n \times p_n$ matrix, and $\varepsilon=(\varepsilon_1,...,\varepsilon_n)'$ consists of independent and identically distributed variables with $E(\varepsilon_1)=0$ and $Var(\varepsilon_1)=\sigma^2$. Suppose $\hat \sigma ^2$ is the estimator of $\sigma^2$. Let

$$\hat \sigma ^2=\frac{\left\|Y_n-X_n\hat\beta\right\|^2}{n-p_n}.$$

How to prove that $\hat \sigma^2$ consistent for $\sigma^2$?

||cite|improve this question
$\endgroup$
  • $\begingroup$ Can you assume normality of the error term $\epsilon$? $\endgroup$ – V. Vancak Nov 16 '16 at 10:21
  • $\begingroup$ No. I cannot assume normality of $\varepsilon$ $\endgroup$ – Vivian Zhang Nov 16 '16 at 23:07
  • $\begingroup$ If I assume normality of the error term, how can I use the assumption to prove that $\hat \sigma^2$ is consistent for $\sigma^2$? $\endgroup$ – Vivian Zhang Nov 28 '16 at 20:22
  • $\begingroup$ I've edited the answer $\endgroup$ – V. Vancak Dec 1 '16 at 12:31
0
$\begingroup$

If you cannot assume normality then $\hat{e}_i = y_i - x_i'\hat{\beta} = e_i+x_i'\beta - x_i'\hat{\beta}=e_i+x_i'(\beta - \hat{\beta})$, recall that $\hat{\beta}_n \xrightarrow{p}\beta$, thus \begin{align} \frac{1}{n-p}||Y-X\hat{\beta}||^2 &= \frac{1}{n-p}\sum_{i=1}^n\hat{e}_i^2\\ &= \frac{1}{n-p}(\sum_{i=1}^n e^2_i + 2\sum_{i=1}^n e_ix_i'(\beta-\hat{\beta}) + (\beta-\hat{\beta})'\sum_{i=1}^n (x_ix_i')(\beta-\hat{\beta})), \end{align} where the second and the third terms converge in probability to $0$ and the first term goes to $\mathbb E{e_i^2}=\sigma^2$.

[1] Such a proof can be found in Econometrics by Bruce E. Hansen. If you can assume normality of the error term, then you can use the $\chi ^2$ distribution to prove consistency which can be slightly easier.

If you can assume normality then the proof becomes much easier as $$ \mathbb{E}\hat{\sigma}^2 = \sigma^2, $$ then the MSE of $\hat{\sigma}^2$ equals its variance, i.e., \begin{align} \lim_{n\to \infty} MSE(\hat{\sigma}^2) &= \lim_{n\to \infty} Var(\frac{\sigma^2}{n-p} \frac{||Y-X\hat{\beta}||^2}{\sigma^2})\\ & =\lim_{n\to \infty}\frac{\sigma^4}{(n-p)^2} Var( \frac{||Y-X\hat{\beta}||^2}{\sigma^2})\\ & = \lim_{n\to \infty}\frac{2(n-p)\sigma^4}{(n-p)^2}\\ &= \lim_{n\to \infty}\frac{2\sigma^4}{(n-p)}=0. \end{align}

||cite|improve this answer
$\endgroup$
  • $\begingroup$ I cannot assume normality. However, I can prove $\hat \sigma^2$ is unbiased estimator for $\sigma^2$. In order to prove the consistency, I need to prove $\lim Var(\hat \sigma^2)=0$. I stuck here. $\endgroup$ – Vivian Zhang Nov 18 '16 at 21:05
  • $\begingroup$ I cannot understand the equation. For the first step, why $\lim_{n \rightarrow \infty} MSE(\hat \sigma^2)=\lim_{n \rightarrow \infty} Var (\frac{\sigma^2}{n-p} \left \| Y_n-X_n\hat \beta \right \|^2)$ ?? where does $\sigma^2$ come from? $\endgroup$ – Vivian Zhang Dec 1 '16 at 19:00
  • $\begingroup$ Sorry, forgot to add the $\sigma^2$ in the denominator too. Anyhow, it is just a "trick" to get the $\chi^2$ distribution and then I used the fact that a variance of $\chi^2_{(n)}$ equals $2n$.. For more details on the you can see this answer. $\endgroup$ – V. Vancak Dec 1 '16 at 22:01
  • $\begingroup$ Thanks for the hint. But in your case, X is followed normal distribution. Does the result still work for my case? Cause in my case, X is not said to be follow normal distribution. $\endgroup$ – Vivian Zhang Dec 7 '16 at 17:53
  • $\begingroup$ No. The chi distribution is a result of the normal assumption. for non normal noise, you can use the first proof. It does not assumes any parametric structure of $\epsilon$. $\endgroup$ – V. Vancak Dec 7 '16 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.