How can we derive the pseudo inverse of a matrix from its Singular value decomposition? For a matrix $M$ with its singular value decomposition $UΣV^T$, the pseudo inverse of $M$, i.e., $M^+$ is $VΣ^+U^T$. 


*

*How can I derive the pseudo inverse(Moore–Penrose) $M^+$ from the singular value decomposition of a matrix $M$? 

*From SVD, we know that $Σ$ is a diagonal matrix which contains the square roots of the eigen values of both $MM^T$ and $M^TM$ whereas $Σ^+$ is formed by replacing the non-zero diagonal elements of $Σ$ by its reciprocal. The diagonal matrix Σ is not always full rank so I assume that $ΣΣ^+$ cannot always be an Identity Matrix. How is it possible to prove that the pseudo inverse of $M$, i.e., $M^+$=$VΣ^+U^T$ holds when $ΣΣ^+$ cannot be reduced to Identity Matrix? Is there any other approach?

 A: $M^+=V\Sigma^+U^\ast$ (i.e. $M^+=V\Sigma^+U^\top$ in the real case), where $\Sigma^+$ is a rectangular diagonal matrix whose size is identical to the size of $\Sigma^\top$. The $i$-th main diagonal entry of $\Sigma^+$ is $\sigma_i^{-1}$ if the $i$-th singular value $\sigma_i$ of $M$ is positive, otherwise the diagonal entry is zero.
You may simply prove that $M^+$ is indeed the Moore-Penrose pseudoinverse of $M$ by showing that it satisfies the four defining properties of Moore-Penrose pseudoinverse, namely, both $MM^+$ and $M^+M$ are Hermitian (or real symmetric in your case), $MM^+M=M$ and $M^+MM^+=M^+$.
A: This is easier to deal with in dyadic (bra-ket) notation.
Write the SVD of $A$ as
$$A=UDV^\dagger = \sum_k d_k u_k v_k^\dagger,$$
where $u_k$ (the left singular vectors) are the columns of $U$, $v_k$ (the right singular vectors) the columns of $V$, and $d_k\equiv D_{kk}>0$ the non-zero singular values.
The pseudo-inverse is then simply
$$A^+ = \sum_k \frac{1}{d_k} v_k u_k^\dagger.$$
You switch left and right singular vectors and take the reciprocal of the singular values.
You can then also observe directly how
$$A^+ A = \sum_k v_k v_k^\dagger = VV^\dagger = \text{projector onto the support of } A, \\
AA^+ = \sum_k u_k u_k^\dagger = UU^\dagger = \text{projector onto the range of }A.$$
(I posted the same answer here).
