For a matrix $M$ with its singular value decomposition $UΣV^T$, the pseudo inverse of $M$, i.e., $M^+$ is $VΣ^+U^T$.

  1. How can I derive the pseudo inverse(Moore–Penrose) $M^+$ from the singular value decomposition of a matrix $M$?
  2. From SVD, we know that $Σ$ is a diagonal matrix which contains the square roots of the eigen values of both $MM^T$ and $M^TM$ whereas $Σ^+$ is formed by replacing the non-zero diagonal elements of $Σ$ by its reciprocal. The diagonal matrix Σ is not always full rank so I assume that $ΣΣ^+$ cannot always be an Identity Matrix. How is it possible to prove that the pseudo inverse of $M$, i.e., $M^+$=$VΣ^+U^T$ holds when $ΣΣ^+$ cannot be reduced to Identity Matrix? Is there any other approach?

$M^+=V\Sigma^+U^\ast$ (i.e. $M^+=V\Sigma^+U^\top$ in the real case), where $\Sigma^+$ is a rectangular diagonal matrix whose size is identical to the size of $\Sigma^\top$. The $i$-th main diagonal entry of $\Sigma^+$ is $\sigma_i^{-1}$ if the $i$-th singular value $\sigma_i$ of $M$ is positive, otherwise the diagonal entry is zero.

You may simply prove that $M^+$ is indeed the Moore-Penrose pseudoinverse of $M$ by showing that it satisfies the four defining properties of Moore-Penrose pseudoinverse, namely, both $MM^+$ and $M^+M$ are Hermitian (or real symmetric in your case), $MM^+M=M$ and $M^+MM^+=M^+$.

  • $\begingroup$ While proving those four properties of Moore-Penrose pseudo inverse, we reach a point where we need to reduce $ΣΣ^+$ to $I$. Is there other ways? $\endgroup$ – kungfu Oct 30 '13 at 7:47
  • $\begingroup$ @kungfu Would you please show me why you need to show that $\Sigma\Sigma^+=I$? To prove the four defining properties, you shouldn't need the equality $\Sigma\Sigma^+=I$ and the equality is not true in general. $\endgroup$ – user1551 Oct 30 '13 at 9:08
  • $\begingroup$ Well, if i want to verify that the pseudo inverse follows property 1 of Moore Penrose Pseudo inverse, i.e., $MM^+M$ = $M$ Starting from Left hand side, $MM^+M$ = $(UΣV^T)(VΣ^+U^T)(UΣV^T)$ = $UΣ(V^TV)Σ^+(U^TU)ΣV^T$ = $UΣΣ^+ΣV^T$ How do i move forward here after? $\endgroup$ – kungfu Oct 30 '13 at 10:25
  • $\begingroup$ @kungfu You may prove that $\Sigma\Sigma^+\Sigma=\Sigma$. Suppose $\Sigma$ is $m\times n$ and it has rank $k$. If $m\ge n$, write $\Sigma=\pmatrix{S_{k\times k}&0_{k\times(n-k)}\\ 0_{(n-k)\times k}&0_{(n-k)\times(n-k)}\\ 0_{(m-n)\times k}&0_{(m-n)\times(n-k)}}$ where $S$ is an invertible diagonal matrix. So, you may directly verify that $\Sigma\Sigma^+\Sigma=\Sigma$. The proof is similar if $m<n$. $\endgroup$ – user1551 Oct 30 '13 at 10:37
  • $\begingroup$ Could you please explain your representation of $Σ$? $\endgroup$ – kungfu Oct 30 '13 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.