Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Assume first $A$ has independent columns. We have a projection matrix $P$ on the range of $A$, i.e. $P_A = A(A^\top A)^{-1}A^\top$. Consider now a matrix $B$ which has the same range as $A$. We have $B = AC$, where $C$ is an invertible matrix. We also have the following.

$P_B = B(B^\top B)^{-1}B^\top = AC(C^\top A^\top AC )^{-1}C^\top A^\top = ACC^{-1}(A^\top A)^{-1} {C^{\top}}^{-1}C^\top A = P_A$

I am interested in having a similar result for the case where A does not have independent columns. Now, $A^\top A$ isn't invertible and the projection matrix becomes $P_A = A(A^\top A)^{\dagger}A^\top$, where we denote by $(\cdot)^\dagger$ the Moore-Penrose pseudoinverse. Consider now two cases:

  1. $B$ also has dependent columns and $B = AC$ for some invertible $C$.
  2. $B$ has independent columns and $B = AC$ for a skinny rectangular matrix $C$ such that $C^\top C = I$, i.e. $C$ selects a minimal subset of columns of $A$ to span the range. Edit: The columns of C are columns of the identity matrix.

In each case, I would like a proof that $A(A^\top A)^{\dagger}A^\top = B(B^\top B)^{\dagger}B^\top$. The problem with the above technique is that I can't simply pull $C$ out of $(C^\top A^\top AC )^{\dagger}$.

Edit: I see that the matrix $A(A^\top A)^{\dagger}A^\top = A A^\dagger$, but again, I don't see how to show that $A A^\dagger = AC (AC)^\dagger$.

Edit: I can also see the following justification: $P_A$ is the projection matrix on the range of $A$, and $P_B$ on the range of $B$. Since the range is the same, they also have to be the same by uniqueness of projection (@martini). But this is ugly, because we have to interpret the matrices as projections. Isn't there an $algebraic$ way to justify this?

share|cite|improve this question
For the case where the columns are not independent, I dont'think $P_A=A(A^{\top} A)^{\dagger} A^{\top}$. Example: if $A=\matrix{\(1 & 1 \)}$, $A(A^{\top} A)^{\dagger} A^{\top}=(4)$, and this is not a projection. – francis-jamet Oct 4 '12 at 13:34
Don't we have $P_A = AA^\dagger$? This has the same range as $A$ and fullfills $AA^\dagger AA^\dagger = AA^\dagger$ and $(AA^\dagger)^\top = AA^\dagger$. – martini Oct 4 '12 at 13:37
Doesn't all this follow from the uniqueness of projectors? – martini Oct 4 '12 at 13:41
$A(A^{\mathsf{T}}A)^{-1}A^{\mathsf{T}}=I$, so I'm not sure you meant that. – Matthew Pressland Oct 4 '12 at 13:43
@MattPressland: For the case where the columns are independent, $A$ is not necessarily a square matrix. – francis-jamet Oct 4 '12 at 13:51
up vote 1 down vote accepted

If $A^{\top}$ is the transpose of $A$, we may have $A \neq 0$ and $P_A=0$. So, $P_A$ is not the projection on the range of $A$ in this case.

Example: If $A=\left( \matrix {1 & 1+i \\ i &i-1} \right)$, $A^{\top} A=0$, so $(A^{\top}A)^{\dagger}=0$ and $P_A=0$, but $A \neq0$.

So, we suppose $A^{\top}$ is the Hermitian transpose (or conjugate transpose). We have $P_A P_A=P_A$ and $P_A^{\top}=P_A$.

  1. If $B=AC$ with $C$ invertible. We have $A^{\top}A(A^{\top}A)^{\dagger}A^{\top}A=A^{\top}A$. So, $$P_B P_A P_B = B(B^{\top}B)^{\dagger}C^{\top}A^{\top}P_A AC (B^{\top}B)^{\dagger}B $$ $$=B(B^{\top}B)^{\dagger}C^{\top}A^{\top}AC (B^{\top}B)^{\dagger}B=P_B P_B=P_B $$

As $A=BC^{-1}$, we have $P_A P_B P_A=P_A$ too.

If $M$ is a matrix, $M^{\top}M=0 \implies M=0$.

If $M=(Id-P_B)P_A$, $M^{\top}M=P_A(Id-P_B)P_A=P_A-P_A P_BP_A=0$, so $M=0$ and $P_A-P_BP_A=0$, and $P_A=P_B P_A$. By hermitian transposition, $P_A=P_A P_B$.

We have $P_B=P_A P_B=P_B P_A$, so $P_A=P_B$.

share|cite|improve this answer
Thank you for the excellent answer. Actually, this also solves point 2, since invertability of $C$ isn't required. Because the span of $A$ and $B$ is the same, we can write $A(A^\top A)^\dagger A^\top = ACX = BX$ for some $X$. Then $P_A P_B P_A = X^\top B^\top B (B^\top B)^\dagger B^\top B X = X^\top B^\top B X = P_A P_A = P_A$. – ziutek Oct 8 '12 at 16:34

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.