In the setting of multiple linear regression where $Y_i = \beta_0 + \beta_1*X_{i1} + ... \beta_p*X_{ip} + \epsilon_i$ and $\epsilon_i \sim N(0, \sigma^2)$ and $Cov(\epsilon_i, \epsilon_j) = 0$. I've been told that unnecessary covariates will increase the variance of the regression coefficients i.e increase $Var(\hat{\beta}) = \sigma^2(X'X)^{-1}$, as they waste degrees of freedom without contributing to the regression sum of squares $B'X'Y - n\bar{Y}^2$. Why is this? Can someone elaborate on this?

Also, $Var(\hat{Y})$ will also increase as number of covariates goes up. Why is this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.