# How can we derive pseudo inverse of a matrix from its Singular value decomposition?

For a matrix $M$ with its singular value decomposition $UΣV^T$, the pseudo inverse of $M$, i.e., $M^+$ is $VΣ^+U^T$.

1. How can I derive the pseudo inverse(Moore–Penrose) $M^+$ from the singular value decomposition of a matrix $M$?
2. From SVD, we know that $Σ$ is a diagonal matrix which contains the square roots of the eigen values of both $MM^T$ and $M^TM$ whereas $Σ^+$ is formed by replacing the non-zero diagonal elements of $Σ$ by its reciprocal. The diagonal matrix Σ is not always full rank so I assume that $ΣΣ^+$ cannot always be an Identity Matrix. How is it possible to prove that the pseudo inverse of $M$, i.e., $M^+$=$VΣ^+U^T$ holds when $ΣΣ^+$ cannot be reduced to Identity Matrix? Is there any other approach?

$M^+=V\Sigma^+U^\ast$ (i.e. $M^+=V\Sigma^+U^\top$ in the real case), where $\Sigma^+$ is a rectangular diagonal matrix whose size is identical to the size of $\Sigma^\top$. The $i$-th main diagonal entry of $\Sigma^+$ is $\sigma_i^{-1}$ if the $i$-th singular value $\sigma_i$ of $M$ is positive, otherwise the diagonal entry is zero.
You may simply prove that $M^+$ is indeed the Moore-Penrose pseudoinverse of $M$ by showing that it satisfies the four defining properties of Moore-Penrose pseudoinverse, namely, both $MM^+$ and $M^+M$ are Hermitian (or real symmetric in your case), $MM^+M=M$ and $M^+MM^+=M^+$.
• While proving those four properties of Moore-Penrose pseudo inverse, we reach a point where we need to reduce $ΣΣ^+$ to $I$. Is there other ways? – kungfu Oct 30 '13 at 7:47
• @kungfu Would you please show me why you need to show that $\Sigma\Sigma^+=I$? To prove the four defining properties, you shouldn't need the equality $\Sigma\Sigma^+=I$ and the equality is not true in general. – user1551 Oct 30 '13 at 9:08
• Well, if i want to verify that the pseudo inverse follows property 1 of Moore Penrose Pseudo inverse, i.e., $MM^+M$ = $M$ Starting from Left hand side, $MM^+M$ = $(UΣV^T)(VΣ^+U^T)(UΣV^T)$ = $UΣ(V^TV)Σ^+(U^TU)ΣV^T$ = $UΣΣ^+ΣV^T$ How do i move forward here after? – kungfu Oct 30 '13 at 10:25
• @kungfu You may prove that $\Sigma\Sigma^+\Sigma=\Sigma$. Suppose $\Sigma$ is $m\times n$ and it has rank $k$. If $m\ge n$, write $\Sigma=\pmatrix{S_{k\times k}&0_{k\times(n-k)}\\ 0_{(n-k)\times k}&0_{(n-k)\times(n-k)}\\ 0_{(m-n)\times k}&0_{(m-n)\times(n-k)}}$ where $S$ is an invertible diagonal matrix. So, you may directly verify that $\Sigma\Sigma^+\Sigma=\Sigma$. The proof is similar if $m<n$. – user1551 Oct 30 '13 at 10:37
• Could you please explain your representation of $Σ$? – kungfu Oct 30 '13 at 14:03