0
$\begingroup$

$y=X\beta+u$ where $u \sim N(0,\Sigma)$ and $\Sigma$ is symmetric & idempotent.
$X: n*k$, $y:n*1$, $\beta=k*1$, $u:n*1$ vector.
Suppose you apply LS(least square) to the model.
Calculate the covariance matrix of $\hat{\beta}$.

Since $\hat{\beta}=(X'X)^{-1}X'y$,
$E(\hat{\beta})=\beta$ so $\hat{\beta}$ is an unbiased estimator,
$Cov(\hat{\beta})=E(\hat{\beta}-E(\hat{\beta}))(\hat{\beta}-E(\hat{\beta}))'=E[(X'X)^{-1}X'uu'X(X'X)^{-1}]=\Sigma (X'X)^{-1}$
Is it right?

$\endgroup$
2
  • $\begingroup$ You might want to look at this, since $\hat{\beta}$ is just a linear transformation of $y$. $\endgroup$ Commented Oct 15, 2013 at 11:59
  • $\begingroup$ @StefanHansen I think I skipped too many parts so now I added them. $\endgroup$
    – dont
    Commented Oct 15, 2013 at 12:00

1 Answer 1

2
$\begingroup$

you may use this property: $Cov\left( Ay \right)=ACov\left( y \right)A'$ where $A={{\left( {X}'X \right)}^{-1}}{X}',Cov\left( y \right)=\text{ }\!\!\Sigma\!\!\text{ }$

so ${{\left( {X}'X \right)}^{-1}}{X}'\text{ }\!\!\Sigma\!\!\text{ }X{{\left( {X}'X \right)}^{-1}}$.

$\endgroup$
7
  • $\begingroup$ How does it simplify to $\Sigma (X'X)^{-1}$? $\endgroup$ Commented Oct 15, 2013 at 12:33
  • $\begingroup$ But $AB$ does not equal $BA$ in general for matrices $A$ and $B$, so you can't move $\Sigma$ to the front. $\endgroup$ Commented Oct 15, 2013 at 12:41
  • $\begingroup$ thanks it escaped my notice. So we get the conclusion $Cov={{\left( {X}'X \right)}^{-1}}{X}'\text{ }\Sigma \text{ }X{{\left( {X}'X \right)}^{-1}}$ and it cant be written as $(X'X)^{-1}$ $\endgroup$
    – mert
    Commented Oct 15, 2013 at 12:53
  • $\begingroup$ Without further assumptions on $\Sigma$, I don't think we can reduce it any more than $(X'X)^{-1} X'\Sigma X(X'X)^{-1}$. $\endgroup$ Commented Oct 15, 2013 at 12:56
  • $\begingroup$ The only assumption i know is $\text{ }\!\!\Sigma\!\!\text{ }={{\sigma }^{2}}I$ when we get ${{\sigma }^{2}}{{\left( {X}'X \right)}^{-1}}$ $\endgroup$
    – mert
    Commented Oct 15, 2013 at 12:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .