Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm going through the article on orthogonal projections on Wikipedia and it reads

If $u_1,\dots,u_k$ is a basis and $A$ is matrix with these vectors as columns then the projection is $$P_A=A(A^TA)^{-1}A^T$$

What if $u_1,\dots,u_k$ are not linearly independent? Does this not still form a projection matrix onto the column space of $A$? Is it because then $(A^TA)^{-1}$ fails to be invertible?

share|cite|improve this question
Note that it says "If $u_1, ..., u_k$ is a (not necessarily orthonormal) basis", meaning that $u_1, ..., u_k$ still must be a basis for this to hold. – Suugaku Apr 16 '13 at 6:43
@Suugaku No I know, I just don't know why it has to be a basis – crf Apr 16 '13 at 6:45
If it's not a basis, then $(A^T A)^{-1}$ doesn't make sense. Or does it? Check! Can $A^T A$ be invertible otherwise? – Suugaku Apr 16 '13 at 6:54
@crf : it is not hard to show $A$ and $A^T A$ have the same null space. If $A$ has linearly independent columns, then by the rank-nullity theorem, the null space of $A$ is trivial, so $A^T A$ is nonsingular. If $A$ does not have linearly independent columns (say, $A$ is a matrix of zeros), then $A^T A$ will be singular. – Stefan Smith Apr 16 '13 at 23:29
up vote 2 down vote accepted

Yes. If the columns of $A$ are not linealy independent then $\text{det} (A)=0$, thus $\text{det}(A\cdot A^T)=\text{det}(A)^2=0$ and so $A\cdot A^T$ is not invertible.

share|cite|improve this answer

At a crucial step in the derivation of the of the projection matrix onto the column space of $A$ we take the inverse of $AA^t$. But how do we know this matrix is invertible? It is invertible, because the columns of A, the vectors $u_{i}$ , were assumed to be linearly independent.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.