$y=X\beta+u$ where $u \sim N(0,\Sigma)$ and $\Sigma$ is symmetric & idempotent.
$X: n*k$, $y:n*1$, $\beta=k*1$, $u:n*1$ vector.
Suppose you apply LS(least square) to the model.
Calculate the covariance matrix of $\hat{\beta}$.

Since $\hat{\beta}=(X'X)^{-1}X'y$,
$E(\hat{\beta})=\beta$ so $\hat{\beta}$ is an unbiased estimator,
$Cov(\hat{\beta})=E(\hat{\beta}-E(\hat{\beta}))(\hat{\beta}-E(\hat{\beta}))'=E[(X'X)^{-1}X'uu'X(X'X)^{-1}]=\Sigma (X'X)^{-1}$
Is it right?

  • $\begingroup$ You might want to look at this, since $\hat{\beta}$ is just a linear transformation of $y$. $\endgroup$ – Stefan Hansen Oct 15 '13 at 11:59
  • $\begingroup$ @StefanHansen I think I skipped too many parts so now I added them. $\endgroup$ – dont Oct 15 '13 at 12:00

you may use this property: $Cov\left( Ay \right)=ACov\left( y \right)A'$ where $A={{\left( {X}'X \right)}^{-1}}{X}',Cov\left( y \right)=\text{ }\!\!\Sigma\!\!\text{ }$

so ${{\left( {X}'X \right)}^{-1}}{X}'\text{ }\!\!\Sigma\!\!\text{ }X{{\left( {X}'X \right)}^{-1}}$.

  • $\begingroup$ How does it simplify to $\Sigma (X'X)^{-1}$? $\endgroup$ – Stefan Hansen Oct 15 '13 at 12:33
  • $\begingroup$ But $AB$ does not equal $BA$ in general for matrices $A$ and $B$, so you can't move $\Sigma$ to the front. $\endgroup$ – Stefan Hansen Oct 15 '13 at 12:41
  • $\begingroup$ thanks it escaped my notice. So we get the conclusion $Cov={{\left( {X}'X \right)}^{-1}}{X}'\text{ }\Sigma \text{ }X{{\left( {X}'X \right)}^{-1}}$ and it cant be written as $(X'X)^{-1}$ $\endgroup$ – mert Oct 15 '13 at 12:53
  • $\begingroup$ Without further assumptions on $\Sigma$, I don't think we can reduce it any more than $(X'X)^{-1} X'\Sigma X(X'X)^{-1}$. $\endgroup$ – Stefan Hansen Oct 15 '13 at 12:56
  • $\begingroup$ The only assumption i know is $\text{ }\!\!\Sigma\!\!\text{ }={{\sigma }^{2}}I$ when we get ${{\sigma }^{2}}{{\left( {X}'X \right)}^{-1}}$ $\endgroup$ – mert Oct 15 '13 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.