1
$\begingroup$

I am trying to derive the finite sample variance of an OLS estimator when the regressor is also random. More concretely, I am looking at the following case:

$$ Y_i = \beta X_i + \epsilon_i $$ where $$ X_i \sim \mathcal{N}(0, \sigma^2_x) \\ \epsilon_i \sim \mathcal{N}(0, \sigma^2_\epsilon) $$

and $X_i$ and $\epsilon_i$ are independent. I know that the OLS estimator $\widehat{\beta}$ is:

$$ \widehat{\beta} = \frac{\sum X_i Y_i}{\sum X^2_i} $$

The $X_i$ and $\epsilon_i$ are i.i.d.

I want to compute the finite sample variance of $\widehat{\beta}$. I have only come across variance results which assume that the regressor $X$ is fixed, i.e., for $\text{Var}(\widehat{\beta}|X)$.

Note, I was able to derive the asymptotic variance of $\widehat{\beta}$. I am stuck on the finite sample case.

Thanks for your help!

$\endgroup$
1

1 Answer 1

1
$\begingroup$

\begin{align} Var(\hat{\beta}) &= E(Var(\hat{\beta}|X)) + Var(E(\hat{\beta}|X))\\ &= E\left( \frac{\sigma^2_{\epsilon}}{\sum X_i^2} \right) + Var(\beta)\\ \end{align} As $X_i$ are i.i.d normal random variables with zero mean and variance of $\sigma^2_{x}$, thus $\sum X_i^2/\sigma_x^2$ is distributed $\chi^2(n)$ hence $(\sum X_i^2/\sigma_x^2)^{-1}$ is distributed Inverse chi squared with with mean of $1/(n-2)$, thus \begin{align} Var(\hat{\beta}) &= E\left( \frac{\sigma^2_{\epsilon}}{\sigma_x^2} \text{Inv-}\chi^2(n) = \right)=\frac{\sigma^2_{\epsilon}}{(n-2)\sigma_x^2} \end{align}

$\endgroup$
2
  • $\begingroup$ @StubbornAtom The conditional variance of $Z$ in your answer is problematic as it should not contain $\sigma_x^2$. $\endgroup$
    – V. Vancak
    Commented Jan 11, 2020 at 13:18
  • $\begingroup$ @Shantanu It is better to open a new thread for the other question so it can be clearly formulated $\endgroup$
    – V. Vancak
    Commented Jan 12, 2020 at 20:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .