I came across this formula in the book Elements of Statistical Learning in Chapter 3.

$$\operatorname{Var}(\hat\beta) = (\mathbf X^T\mathbf X)^{-1}\hat\sigma^2$$

Here, $\hat\beta$ is the estimated parameter vector for a linear regression equation satisfying: $$\mathbf y= \mathbf X\mathbf \beta + \epsilon$$

$\epsilon$ in turn follows a normal distribution with mean 0 and variance $\sigma^2$. $\hat\beta$ follows the normal equation, which has been derived previously in the same chapter: $$ \hat\beta = (\mathbf X^T \mathbf X)^{-1}\mathbf X^T\mathbf y$$

What I understand as per the text is that the $\operatorname{Var}(\hat\beta)$ is the variance-covariance derivation, and for a matrix I can calculate the variance-matrix given values. But I'm getting stuck in this case as $\hat\beta$ is expressed as an equation, which when I use to expand $\hat\beta$ in my derivation, I end up with an equation that I cannot simplify further.

  • 1
    $\begingroup$ it might be helpful if you include your partial derivation. $\endgroup$
    – 311411
    Mar 29 at 19:32

I found the solution to be very well written in the following question: Variance Estimate in linear regression

The key to solving this equation is $\operatorname{Var}(AW) = A\operatorname{Var}(W)A^T$

Marking this question as answered.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.