# Linear independence of projection matrices

Probably it is a rather simple question ( I'm not sure) but I would like to know answer supported by a proof.

Can it be proved that

IF
unit vectors $v_1,v_2,.. v_n$ are linearly independent
THEN
their projection matrices ${v_1}{v_1}^T,{v_2}{v_2}^T,.. {v_n}{v_n}^T$ are also linearly independent?

...and second case..

IF
unit vectors $v_1,v_2,.. v_n$ are linearly dependent
THEN
their projection matrices ${v_1}{v_1}^T,{v_2}{v_2}^T,.. {v_n}{v_n}^T$ are also linearly dependent?

• The first statement is easy to prove, the second is easy to disprove (already in dimension $2$). Try it. – Marc van Leeuwen Oct 21 '16 at 13:33
• @MarcvanLeeuwen Ok, for the second case I've found example $[1, 0]^T , [0, 1]^T, [1, 1]^T$, but the first case seems to be harder... – Widawensen Oct 21 '16 at 13:43
• @MarcvanLeeuwen Sometimes saying that something is easy leads to the solution.. maybe I've found a method of proving.. let's suppose that we have vectors linearly independent. If we assume that projection matrices can be linearly dependent then we have their some linear combination equal to 0 matrix. Now we can multiply all matrices by, for example, $v_1$, from this we would have that linear combination of vectors would be equal to zero vector, what is contradiction... Am I right ?.. – Widawensen Oct 21 '16 at 14:05
• @Widawensen - Yes you are right; I just finished writing my answer demonstrating this. – Myridium Oct 21 '16 at 14:10
• @Myridium O.K Let our answers be together : mine in comments, yours as the main one. – Widawensen Oct 21 '16 at 15:40

You've already found a counterexample to the second statement, so I'll just prove the first.

Proof that linear independence of vectors implies linear independence of projections:

Let $v_1,v_2, \dots, v_n$ be linearly independent. Suppose that $P_1, P_2 , \dots , P_n$ are not linearly independent.

Without loss of generality, say that $P_n$ can be expressed as a combination of the other projections (we can reorder the projections so that this happens): $$P_n = \sum_{i=1}^{n-1} \lambda_i \cdot P_i$$

We would hence expect that a vector projected onto the axis of $v_n$ can be created instead as a combination of the other projections of that vector. Pick an $\mathbf x \in V$ and let $P_i(\mathbf x) = \alpha_i \cdot v_i$. Then: \begin{align}P_n(\mathbf x) &= \sum_{i=1}^{n-1} \lambda_i \cdot P_i(\mathbf x) \\ \alpha_n \cdot v_n &= \sum_{i=1}^{n-1}\lambda_i \cdot (\alpha_i \cdot v_i) \\ v_n &= \frac 1 {\alpha_n} \sum_{i=1}^{n-1}\lambda_i \cdot (\alpha_i \cdot v_i) \end{align} Provided that $\alpha_n \neq 0$, we have shown that $v_n$ is a linear combination of the other $v_i$. All that is left to do is pick an $\mathbf x$ so that $\alpha_n \neq 0$. This is easy-- pick $\mathbf x = v_n$. This makes $\alpha_n = v_n^T v_n = ||v_n||_2^2$ which is non-zero (because otherwise $v_n$ is the zero vector) and so

$$v_n = \frac 1 {||v_n||_2^2} \sum_{i=1}^{n-1}\lambda_i \cdot (\alpha_i \cdot v_i)$$

This reveals that $v_n$ is a linear combination of the other $v_i$; a contradiction. $\square$

• The proof has some minor issues, though the idea is valid. A linear dependency does not always allow expressing the last one as linear combination of the previous (though one can assume this after permutation); more seriously what is $\alpha_n$ and why should it necessarily be nonzero (on which the proof hinges)? – Marc van Leeuwen Oct 21 '16 at 17:01
• @MarcvanLeeuwen - Thanks for spotting these. It should be fixed now. – Myridium Oct 22 '16 at 1:09
• It is fine now if you remove the superfluous claim that $\alpha_n=1$; linear independence does not imply the vectors have unit length. – Marc van Leeuwen Oct 22 '16 at 6:34
• @MarcvanLeeuwen - Thanks again. Let me know if I've still messed something up. – Myridium Oct 22 '16 at 7:10

The first statement is easy to prove, the second is easy to disprove.

For the first, I'll show that if the projections $v_iv_i^T$ are linearly dependent, then so are the vectors$~v_i$. So by assumption we have scalars $\lambda_1,\ldots,\lambda_n$, not all zero, such that $\sum_i\lambda_i v_iv_i^T=0$. Let $k$ be an index with $\lambda_k\neq0$; we can assume $v_k\neq0$ for otherwise the linear dependency is obvious. Then let $w$ be a vector with $v_k^T(w)\neq 0$ (if this is over the real numbers one can take $w=v_k$, but in any case $v_k^T\neq0$ ensures that such $w$ exists). Then $0=0(w)=\sum_i\lambda_i v_iv_i^T(w)=\sum_i(\lambda_i v_i^T(w))v_i$, which is a relation with $\lambda_kv_k^T(w)$ a nonzero coefficient, so it establishes linear dependency.

To disprove the second claim, you can take $v,w,v+w$ for any pair of orthogonal unit vectors $v,w$, and compute their projections explicitly to see that they are linearly independent.

• Additionally I have noticed that using similar argumentation if we have k vectors in n-dimensional space (k<n) (case 1) projection matrices are also linearly independent from Identity Matrix $I$. Am I right ? – Widawensen Oct 23 '16 at 7:20
• @Widawensen: Not really, since the conditions do not guarantee that the matrices $v_iv_i^T$ are linearly independent among themselves, which is necessary for them to be linearly independent with $I$ thrown in as well. But indeed, $I$ can never be a linear combination of less than $n$ operators of rank$~1$, for rank reasons. – Marc van Leeuwen Oct 23 '16 at 7:23
• I meant the first case in the question where the matrices $v_iv_i^T$ are linearly independent among themselves .... – Widawensen Oct 23 '16 at 8:20
• However the argument with the rank is enough good,.... your first remark about easy way of proving the case was very inspiring, thank you, .. maybe you might take a look at the related question math.stackexchange.com/questions/1975788/… and shortly comment whether a single formula could exist for a solution of problem? ( using determinant for example) – Widawensen Oct 23 '16 at 8:36