# Calculate the covariance matrix of $\hat{\beta}$

$y=X\beta+u$ where $u \sim N(0,\Sigma)$ and $\Sigma$ is symmetric & idempotent.
$X: n*k$, $y:n*1$, $\beta=k*1$, $u:n*1$ vector.
Suppose you apply LS(least square) to the model.
Calculate the covariance matrix of $\hat{\beta}$.

Since $\hat{\beta}=(X'X)^{-1}X'y$,
$E(\hat{\beta})=\beta$ so $\hat{\beta}$ is an unbiased estimator,
$Cov(\hat{\beta})=E(\hat{\beta}-E(\hat{\beta}))(\hat{\beta}-E(\hat{\beta}))'=E[(X'X)^{-1}X'uu'X(X'X)^{-1}]=\Sigma (X'X)^{-1}$
Is it right?

• You might want to look at this, since $\hat{\beta}$ is just a linear transformation of $y$. – Stefan Hansen Oct 15 '13 at 11:59
• @StefanHansen I think I skipped too many parts so now I added them. – dont Oct 15 '13 at 12:00

you may use this property: $Cov\left( Ay \right)=ACov\left( y \right)A'$ where $A={{\left( {X}'X \right)}^{-1}}{X}',Cov\left( y \right)=\text{ }\!\!\Sigma\!\!\text{ }$
so ${{\left( {X}'X \right)}^{-1}}{X}'\text{ }\!\!\Sigma\!\!\text{ }X{{\left( {X}'X \right)}^{-1}}$.
• How does it simplify to $\Sigma (X'X)^{-1}$? – Stefan Hansen Oct 15 '13 at 12:33
• But $AB$ does not equal $BA$ in general for matrices $A$ and $B$, so you can't move $\Sigma$ to the front. – Stefan Hansen Oct 15 '13 at 12:41
• thanks it escaped my notice. So we get the conclusion $Cov={{\left( {X}'X \right)}^{-1}}{X}'\text{ }\Sigma \text{ }X{{\left( {X}'X \right)}^{-1}}$ and it cant be written as $(X'X)^{-1}$ – mert Oct 15 '13 at 12:53
• Without further assumptions on $\Sigma$, I don't think we can reduce it any more than $(X'X)^{-1} X'\Sigma X(X'X)^{-1}$. – Stefan Hansen Oct 15 '13 at 12:56
• The only assumption i know is $\text{ }\!\!\Sigma\!\!\text{ }={{\sigma }^{2}}I$ when we get ${{\sigma }^{2}}{{\left( {X}'X \right)}^{-1}}$ – mert Oct 15 '13 at 12:59