2
$\begingroup$

I have the model:

$y=X{\beta}+{\epsilon}$

I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased estimator of the variance.

How do I show that $\hat{\beta}$ and $s^2$ are independent?

$\endgroup$
  • $\begingroup$ The two estimates are in orthogonal subspaces. I think you need to mention that that errors are iid normal. A similar, but simpler situation is that for iid normal data $X_i,$ we have $\bar X$ and $S^2$ are stochastically independent (even though $\bar X$ appears in the definition of $S^2.$ $\endgroup$ – BruceET Jun 1 '15 at 7:57
  • $\begingroup$ @BruceTrumbo I was thinking that I needed to do something to do with covariance? $\endgroup$ – Emma Jun 1 '15 at 8:48
  • $\begingroup$ Be careful with that. Zero covariance (or correlation) implies independence only for normal random variables. Even if errors are are normal, that doesn't mean distributions of $\hat \beta$ and $s^2$ are normal. (Example: For uniform data, $\bar X$ and $S^2$ are not independent.) OK if $\hat \beta$ and $s^2$ are functions of orthogonal sets of normal variates. $\endgroup$ – BruceET Jun 1 '15 at 15:23
  • $\begingroup$ @BruceTrumbo ok but this question is a show that question which normally means there's some algebra to be done and then at the end out pops the answer? Is there any way that this could be done algebraically? $\endgroup$ – Emma Jun 1 '15 at 16:08
  • $\begingroup$ There are a few methods of proving independence, none of them magical. No context provided. Difficult to be helpful. Maybe start with simple linear regression. Write $\bar X$ and $S^2$ in terms of data. Look at ANOVA table. Why are "Regression" and "Error" on different rows? Why is DF(Total) = DF(Regression) + DF(Error)? $\endgroup$ – BruceET Jun 1 '15 at 18:14
1
$\begingroup$

Since $s^2:=\hat\epsilon^T\hat\epsilon/(n-k)$ is a function of the residual vector $\hat\epsilon:=Y-X\hat\beta$, this result follows from the independence of $\hat\beta$ and $Y-X\hat\beta$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.