# About linear dependence in covariance matrix and implications on the variables

If I have a covariance matrix with rank less than full, I can discard of a variable as long as the remaining variables still span the original space.

But let's say I discover that for $3\times 3$ covariance matrix, I get that $r_1=r_2+r_3$ for a matrix row $r$. Does this mean that $X_1=X_2+X_3$?

Thanks.

*Edit: Maybe I should refine it a bit. My question is about the link from dependency in the covariance matrix, which represents only the 2nd order statistics, and the liner dependency (not statistical dependency) of the variables. Thanks.

-
Anyone? The question is really about the link between linear dependency in the covariance matrix, and linear dependency in the random variables. – yoki Nov 22 '12 at 6:03

## 1 Answer

Okay, I found the answer.

Let $\Lambda$ be the covariance matrix for some random vector $X$.

A singular covariance matrix implies there's a nonzero vector $y$ s.t. $y'\Lambda y = 0$. This would indicate that there exists a random variable $Y=y'X$ for which $var(Y)=var(y'X)=y'\Lambda y=0$, hence it is known w.p. 1.

Since $Y=y_0$ w.p.1 we know that there exists some linear combination s.t. $y_0=\sum b_i X_i$ and hence $\frac{y_0}{b_1} - \sum_{i=2}^N \frac{b_i}{b_1}X_i = X_1$, or: one of the variables is a.s. a linear combination of the others.

-