1
$\begingroup$

I am trying to derive the finite sample variance of an OLS estimator when the regressor is also random. More concretely, I am looking at the following case:

$$ Y_i = \beta X_i + \epsilon_i $$ where $$ X_i \sim \mathcal{N}(0, \sigma^2_x) \\ \epsilon_i \sim \mathcal{N}(0, \sigma^2_\epsilon) $$

and $X_i$ and $\epsilon_i$ are independent. I know that the OLS estimator $\widehat{\beta}$ is:

$$ \widehat{\beta} = \frac{\sum X_i Y_i}{\sum X^2_i} $$

The $X_i$ and $\epsilon_i$ are i.i.d.

I want to compute the finite sample variance of $\widehat{\beta}$. I have only come across variance results which assume that the regressor $X$ is fixed, i.e., for $\text{Var}(\widehat{\beta}|X)$.

Note, I was able to derive the asymptotic variance of $\widehat{\beta}$. I am stuck on the finite sample case.

Thanks for your help!

$\endgroup$
1
1
$\begingroup$

\begin{align} Var(\hat{\beta}) &= E(Var(\hat{\beta}|X)) + Var(E(\hat{\beta}|X))\\ &= E\left( \frac{\sigma^2_{\epsilon}}{\sum X_i^2} \right) + Var(\beta)\\ \end{align} As $X_i$ are i.i.d normal random variables with zero mean and variance of $\sigma^2_{x}$, thus $\sum X_i^2/\sigma_x^2$ is distributed $\chi^2(n)$ hence $(\sum X_i^2/\sigma_x^2)^{-1}$ is distributed Inverse chi squared with with mean of $1/(n-2)$, thus \begin{align} Var(\hat{\beta}) &= E\left( \frac{\sigma^2_{\epsilon}}{\sigma_x^2} \text{Inv-}\chi^2(n) = \right)=\frac{\sigma^2_{\epsilon}}{(n-2)\sigma_x^2} \end{align}

$\endgroup$
2
  • $\begingroup$ @StubbornAtom The conditional variance of $Z$ in your answer is problematic as it should not contain $\sigma_x^2$. $\endgroup$ – V. Vancak Jan 11 '20 at 13:18
  • $\begingroup$ @Shantanu It is better to open a new thread for the other question so it can be clearly formulated $\endgroup$ – V. Vancak Jan 12 '20 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.