In the context of simple linear regression, suppose that $\epsilon_i, \ i=1,...,n$ are i.i.d and $ |n^{-1}\sum_{i=1}^{n}x_{i}| \rightarrow |\mu| < \infty$ where n $\rightarrow \infty$ and var(x) = $n^{-1}\sum_{i=1}^{n}(x_i-\overline{x_n})^2\rightarrow \alpha \in \mathbb{R}^{+*}.$

Under this assumption, how can we prove that:

i) $\hat{\beta}_1\overset{a.s.}{\to} {\beta}_1$ and $\hat{\beta}_2\overset{a.s.}{\to} {\beta}_2$?

ii) $\widehat{\sigma^2} \overset{a.s.}{\to} \sigma^2$ when $n \to \infty$?

Here, symbols with a hat on top refer to the least square estimators of the coefficients in $y_i=\beta_1+\beta_2x_i+\epsilon_i$, where $\epsilon_i$ is not assumed to be normal. We assume homoskedasticity and zero expectation and zero correlation for the errors (with variance $\sigma^2$).


1 Answer 1


Observe that $$ \hat{\beta_2}-\beta_2=\frac 1{\sum_{i=1}^n(x_i-\bar{x_n})^2}\sum_{j=1}^n (x_j-\bar{x_n})\left(\varepsilon_j-\frac 1n\sum_{i=1}^n\varepsilon_i\right) $$ Since $\sum_{j=1}^n (x_j-\bar{x_n})=0$ and $n^{-1}\sum_{i=1}^{n}(x_i-\overline{x_n})^2\rightarrow \alpha $, it suffices to prove that $$ \frac 1n\sum_{j=1}^nx_j \varepsilon_j\to 0\mbox{ a.s.} $$ This can be done for example by showing that $\sum_{N\geqslant 1}\mathbb E\left[2^{-2N}\max_{1\leqslant n\leqslant 2^N} \left\lvert x_j \varepsilon_j\right\rvert^2\right]$ is finite.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.