# Multicollinearity: Why does highly correlated columns in the design matrix lead to high variance of the regression coefficient?

I came across the term "Multicollinearity" in statistics, particularly statistics. However, I never really understand mathematically why highly correlated (almost linearly dependent) columns in the design matrix $X$ lead to higher variance of the regression coefficient, given that the $$Var(\hat{\beta}) = \sigma^2 (X^TX)^{-1}$$ ?

Can someone explain to me mathematically the idea behind please ?

• That formula doesn't even make sense if the columns of $X$ are linearly dependent, because then $X^TX$ is not invertible. Multicollinearity in statistics usually refers not to the case where the columns of $X$ are exactly dependent but to the case where the explanatory variables are highly correlated. – symplectomorphic Mar 6 '15 at 16:42
• I meant to say high correlated columns, not perfectly correlated columns. – mynameisJEFF Mar 6 '15 at 16:47

$var(\beta)=\sigma^2(X^TX)^{-1}$
First, note that is the columns of X are linearly dependent, then $X'X$ will not be invertible and have a determinant of 0. And, I won't prove this next part, but if $X'X$ is "close" to being linearly dependent, then the determinant will be "close" to 0.
$A^{-1} = \frac{1}{det(A)}adj(A)$
where $adj(A)$ is the adjugte of the matrix A.
Therefore, as $X'X$ becomes more linearly dependent, its determinant becomes closer to 0, which means the elements of the inverse get larger.