If I have a covariance matrix with rank less than full, I can discard of a variable as long as the remaining variables still span the original space.

But let's say I discover that for $3\times 3$ covariance matrix, I get that $r_1=r_2+r_3$ for a matrix row $r$. Does this mean that $X_1=X_2+X_3$?


*Edit: Maybe I should refine it a bit. My question is about the link from dependency in the covariance matrix, which represents only the 2nd order statistics, and the liner dependency (not statistical dependency) of the variables. Thanks.

  • $\begingroup$ Anyone? The question is really about the link between linear dependency in the covariance matrix, and linear dependency in the random variables. $\endgroup$
    – yoki
    Nov 22, 2012 at 6:03

1 Answer 1


Okay, I found the answer.

Let $\Lambda$ be the covariance matrix for some random vector $X$.

A singular covariance matrix implies there's a nonzero vector $y$ s.t. $y'\Lambda y = 0$. This would indicate that there exists a random variable $Y=y'X$ for which $var(Y)=var(y'X)=y'\Lambda y=0$, hence it is known w.p. 1.

Since $Y=y_0$ w.p.1 we know that there exists some linear combination s.t. $y_0=\sum b_i X_i$ and hence $\frac{y_0}{b_1} - \sum_{i=2}^N \frac{b_i}{b_1}X_i = X_1$, or: one of the variables is a.s. a linear combination of the others.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .