# the difference between statistically independent and linearly independent?

what is the difference between statistically independent and linearly independent concepts?

• Wikipedia has great articles on these two topics Sep 2, 2014 at 18:25
• They are unrelated concepts in different fields of mathematics. Sep 2, 2014 at 18:27
• @TylerHG, I readed the Wiki aricles , but they go deep in the two concepts , so i get lost without having a clear idea :( Sep 2, 2014 at 18:53
• @Learner It's a bit odd to take two concepts whose definitions are totally unrelated to each other and then ask "what's the difference?" If you were thinking that sharing one word in their names was enough to imply some sort of relationship, then you're in for a lot of big surprises while studying mathematics :) Sep 3, 2014 at 12:48
• @rschwieb Both the concepts of linear independence and statistical independence are encountered in the world of statistics, often in close proximity (e.g., one frequently sees "covariance is a measure of linear dependence"). Furthermore, linear independence has clear mathematical implications vis-à-vis statistical independence. I therefore don't agree that they are completely unrelated concepts from different branches of maths. The links and differences between them are frequent sources of confusion in statistics, so I think this question is rather worthwhile. Sep 3, 2014 at 13:46

Consider a simple scenario in which you have two non-zero, non-constant, $n$-dimensional data vectors $\mathbf{X}$ and $\mathbf{Y}$.

They are linearly independent if there is no non-zero scalar $\alpha$ such that

$\alpha \mathbf{X} - \mathbf{Y} = \mathbf{0}$

In other words, there is no non-zero multiplicative constant $\alpha$ that will transform $\mathbf{X}$ into $\mathbf{Y}$. Geometrically, this means that the vectors $\mathbf{X}$ and $\mathbf{Y}$ do not lie on the same line.

The two vectors $\mathbf{X}$ and $\mathbf{Y}$ are statistically independent if and only if their joint probability density is the product of their marginal probability densities, i.e.,

$f(\mathbf{X}, \mathbf{Y}) = f_X(\mathbf{X}) \cdot f_Y(\mathbf{Y})$

This implies

cov$(\mathbf{X}, \mathbf{Y}) = \mathbf{0}$

(though the reverse implication is not true generally).

The two concepts are linked insofar as if the two vectors are not linearly independent then they can also not be statistically independent. For example, if for some non-zero scalar $\alpha$ we have

$\alpha \mathbf{X} = \mathbf{Y}$

then

cov$(\mathbf{X}, \mathbf{Y}) = \text{cov}(\frac{1}{\alpha}\mathbf{Y}, \mathbf{Y}) = \frac{1}{\alpha} \text{var} (\mathbf{Y}) \ne \mathbf{0}$

However, linear independence of $\mathbf{X}$ and $\mathbf{Y}$ does not guarantee statistical independence (it is possible to have cov$(\mathbf{X}, \mathbf{Y}) \ne \mathbf{0}$ even if $\mathbf{X}$ and $\mathbf{Y}$ are linearly independent).

• so does this mean that stochastic (namely statistical) independence implies linear independence, but the converse result does not hold most times? Jan 6, 2022 at 21:14
• @HungerLearn this post shows that linear dependence implies statistical dependence. By equivalency of the contrapositive, we must also have that statistical independence implies linear independence.
– S.C.
Mar 22 at 15:28