The Moore-Penrose inverse of a rectangular matrix is given by: $$ A^{+}=V_{r}S_{r}^{-1}U_{r}^{T} $$ Here I am using the economical size SVD. Where $U_{r}^{T}U_{r}=V_{r}^{T}V_{r}=I_{r}$. My question is

It is known that $(A^{T}A)^{+}=V_{r}S_{r}^{-2}V_{r}^{T}$ so my question is how can we show it instead of proving $(A^{T}A)^{+}(A^{T}A)=I_{r}$?

Edit :

I would like to thank user @cangrejo for his suggesstion which I find useful as it confirms my assumption that this problem can be tackled by using properties of the pseudo-inverse : $$ (A^{T}A)^{+}=A^{+}(A^{+})^{T} $$ However, the wikipedia article en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse under the section properties lists one of the sufficient conditions for $(AB)^{+}=B^{+}A^{+}$ is that $B=A^{T}$ which is similar to my case but how can we prove it to be a sufficient condition to the main result $(AB)^{+}=B^{+}A^{+}$ for which this would help me complete the main objective of finding pseudoinverse of $A^{T}A$? In other words, how do we show that if the expression $B=A^{T}$ is true, then this will allow me to write $(AB)^{+}=B^{+}A^{+}$

I will award the bounty for a complete answer.

  • $\begingroup$ Do the shapes match? $\endgroup$
    – cangrejo
    Jun 9 at 16:48
  • $\begingroup$ Yes $A$ is $m\times n$ rectangular matrix. $\endgroup$
    – SPARSE
    Jun 9 at 16:51
  • $\begingroup$ Are the shapes of $(A^TA)^+$ and $(A^T)^+A^+$ equal? $\endgroup$
    – cangrejo
    Jun 9 at 16:54
  • $\begingroup$ In any case, I think the Wikipedia page might have an answer for you: en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse $\endgroup$
    – cangrejo
    Jun 9 at 16:58
  • 3
    $\begingroup$ Note that in general $(A^T A)^+(A^T A) \neq I$, for example if $A=0$. Instead, one needs to verify the 4 Moore-Penrose conditions $(A^T A)(A^T A)^+(A^T A) = (A^T A)$, $(A^T A)^+(A^T A)(A^T A)^+ = (A^T A)^+$, etc. $\endgroup$
    – Hyperplane
    Jun 11 at 23:13

A general strategy for proving statements about the Moore-Penrose inverse is to (1) use a SVD to reduce statements about general matrices to statements about diagonal matrices, and (2) use the 4 Moore-Penrose conditions:

  1. $X Y X = X$
  2. $Y X Y = Y$
  3. $(XY)^T = XY$
  4. $(YX)^T = Y X$

If $Y$ satisfies these 4 equations, then $Y=X^+$. Note that the pseudo-inverse is always unique. So, all we need to do to prove $B=A^T\implies (AB)^+ = B^+ A^+$ is to verify that $Y=B^+A^+$ satisfies these 4 equations for $X=AB$.

Given the economical SVD $A=USV^T$, and $B=A^T = V^T S^T U = V^T S U$ we have

  • $A^+ = VS^{-1}U^T$
  • $B^+ = US^{-1}V^T$
  • $AB = (USV^T)(VSU^T) = US^2 U^T$
  • $B^+A^+ = (US^{-1}V^T)(VS^{-1}U^T) = US^{-2}U^T$

Plugging these into the 4 MP conditions yields straightforward computations

  1. $(AB)(B^+A^+)(AB) = \big(US^2U^T\big)\big(U(S^{-2}U^T\big)\big(US^2U^T\big) = US^2 U^T = AB$

  2. $(B^+A^+)(AB)(B^+A^+) = \big(U(S^{-2}U^T\big)\big(US^2U^T\big)\big(U(S^{-2}U^T\big) = US^{-2} U^T = B^+A^+$

  3. $\big((AB)(B^+A^+)\big)^T = \Big(\big(US^2U^T\big)\big(U(S^{-2}U^T\big)\Big)^T = (UU^T)^T = UU^T= (AB)(B^+A^+)$

  4. $\big((B^+A^+)(AB)\big)^T = \Big(\big(U(S^{-2}U^T\big)\big(US^2U^T\big)\Big)^T = (UU^T)^T = UU^T= (B^+A^+)(AB)$

Notice how, due to the cancellation of the (semi-)orthogonal matrices, all statements are effectively reduced to statements about diagonal matrices.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.