Probably it is a rather simple question ( I'm not sure) but I would like to know answer supported by a proof.

Can it be proved that

unit vectors $v_1,v_2,.. v_n$ are linearly independent
their projection matrices ${v_1}{v_1}^T,{v_2}{v_2}^T,.. {v_n}{v_n}^T$ are also linearly independent?

...and second case..

unit vectors $v_1,v_2,.. v_n$ are linearly dependent
their projection matrices ${v_1}{v_1}^T,{v_2}{v_2}^T,.. {v_n}{v_n}^T$ are also linearly dependent?

  • $\begingroup$ The first statement is easy to prove, the second is easy to disprove (already in dimension $2$). Try it. $\endgroup$ – Marc van Leeuwen Oct 21 '16 at 13:33
  • $\begingroup$ @MarcvanLeeuwen Ok, for the second case I've found example $[1, 0]^T , [0, 1]^T, [1, 1]^T$, but the first case seems to be harder... $\endgroup$ – Widawensen Oct 21 '16 at 13:43
  • $\begingroup$ @MarcvanLeeuwen Sometimes saying that something is easy leads to the solution.. maybe I've found a method of proving.. let's suppose that we have vectors linearly independent. If we assume that projection matrices can be linearly dependent then we have their some linear combination equal to 0 matrix. Now we can multiply all matrices by, for example, $v_1$, from this we would have that linear combination of vectors would be equal to zero vector, what is contradiction... Am I right ?.. $\endgroup$ – Widawensen Oct 21 '16 at 14:05
  • $\begingroup$ @Widawensen - Yes you are right; I just finished writing my answer demonstrating this. $\endgroup$ – Myridium Oct 21 '16 at 14:10
  • $\begingroup$ @Myridium O.K Let our answers be together : mine in comments, yours as the main one. $\endgroup$ – Widawensen Oct 21 '16 at 15:40

You've already found a counterexample to the second statement, so I'll just prove the first.

Proof that linear independence of vectors implies linear independence of projections:

Let $v_1,v_2, \dots, v_n$ be linearly independent. Suppose that $P_1, P_2 , \dots , P_n$ are not linearly independent.

Without loss of generality, say that $P_n$ can be expressed as a combination of the other projections (we can reorder the projections so that this happens): $$P_n = \sum_{i=1}^{n-1} \lambda_i \cdot P_i$$

We would hence expect that a vector projected onto the axis of $v_n$ can be created instead as a combination of the other projections of that vector. Pick an $\mathbf x \in V$ and let $P_i(\mathbf x) = \alpha_i \cdot v_i$. Then: $$\begin{align}P_n(\mathbf x) &= \sum_{i=1}^{n-1} \lambda_i \cdot P_i(\mathbf x) \\ \alpha_n \cdot v_n &= \sum_{i=1}^{n-1}\lambda_i \cdot (\alpha_i \cdot v_i) \\ v_n &= \frac 1 {\alpha_n} \sum_{i=1}^{n-1}\lambda_i \cdot (\alpha_i \cdot v_i) \end{align}$$ Provided that $\alpha_n \neq 0$, we have shown that $v_n$ is a linear combination of the other $v_i$. All that is left to do is pick an $\mathbf x$ so that $\alpha_n \neq 0$. This is easy-- pick $\mathbf x = v_n$. This makes $\alpha_n = v_n^T v_n = ||v_n||_2^2$ which is non-zero (because otherwise $v_n$ is the zero vector) and so

$$v_n = \frac 1 {||v_n||_2^2} \sum_{i=1}^{n-1}\lambda_i \cdot (\alpha_i \cdot v_i)$$

This reveals that $v_n$ is a linear combination of the other $v_i$; a contradiction. $\square$

  • 2
    $\begingroup$ The proof has some minor issues, though the idea is valid. A linear dependency does not always allow expressing the last one as linear combination of the previous (though one can assume this after permutation); more seriously what is $\alpha_n$ and why should it necessarily be nonzero (on which the proof hinges)? $\endgroup$ – Marc van Leeuwen Oct 21 '16 at 17:01
  • $\begingroup$ @MarcvanLeeuwen - Thanks for spotting these. It should be fixed now. $\endgroup$ – Myridium Oct 22 '16 at 1:09
  • 1
    $\begingroup$ It is fine now if you remove the superfluous claim that $\alpha_n=1$; linear independence does not imply the vectors have unit length. $\endgroup$ – Marc van Leeuwen Oct 22 '16 at 6:34
  • $\begingroup$ @MarcvanLeeuwen - Thanks again. Let me know if I've still messed something up. $\endgroup$ – Myridium Oct 22 '16 at 7:10

The first statement is easy to prove, the second is easy to disprove.

For the first, I'll show that if the projections $v_iv_i^T$ are linearly dependent, then so are the vectors$~v_i$. So by assumption we have scalars $\lambda_1,\ldots,\lambda_n$, not all zero, such that $\sum_i\lambda_i v_iv_i^T=0$. Let $k$ be an index with $\lambda_k\neq0$; we can assume $v_k\neq0$ for otherwise the linear dependency is obvious. Then let $w$ be a vector with $v_k^T(w)\neq 0$ (if this is over the real numbers one can take $w=v_k$, but in any case $v_k^T\neq0$ ensures that such $w$ exists). Then $0=0(w)=\sum_i\lambda_i v_iv_i^T(w)=\sum_i(\lambda_i v_i^T(w))v_i$, which is a relation with $\lambda_kv_k^T(w)$ a nonzero coefficient, so it establishes linear dependency.

To disprove the second claim, you can take $v,w,v+w$ for any pair of orthogonal unit vectors $v,w$, and compute their projections explicitly to see that they are linearly independent.

  • $\begingroup$ Additionally I have noticed that using similar argumentation if we have k vectors in n-dimensional space (k<n) (case 1) projection matrices are also linearly independent from Identity Matrix $I$. Am I right ? $\endgroup$ – Widawensen Oct 23 '16 at 7:20
  • $\begingroup$ @Widawensen: Not really, since the conditions do not guarantee that the matrices $v_iv_i^T$ are linearly independent among themselves, which is necessary for them to be linearly independent with $I$ thrown in as well. But indeed, $I$ can never be a linear combination of less than $n$ operators of rank$~1$, for rank reasons. $\endgroup$ – Marc van Leeuwen Oct 23 '16 at 7:23
  • $\begingroup$ I meant the first case in the question where the matrices $v_iv_i^T$ are linearly independent among themselves .... $\endgroup$ – Widawensen Oct 23 '16 at 8:20
  • $\begingroup$ However the argument with the rank is enough good,.... your first remark about easy way of proving the case was very inspiring, thank you, .. maybe you might take a look at the related question math.stackexchange.com/questions/1975788/… and shortly comment whether a single formula could exist for a solution of problem? ( using determinant for example) $\endgroup$ – Widawensen Oct 23 '16 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.