# How can we derive the pseudo inverse of a matrix from its Singular value decomposition?

For a matrix $M$ with its singular value decomposition $UΣV^T$, the pseudo inverse of $M$, i.e., $M^+$ is $VΣ^+U^T$.

1. How can I derive the pseudo inverse(Moore–Penrose) $M^+$ from the singular value decomposition of a matrix $M$?
2. From SVD, we know that $Σ$ is a diagonal matrix which contains the square roots of the eigen values of both $MM^T$ and $M^TM$ whereas $Σ^+$ is formed by replacing the non-zero diagonal elements of $Σ$ by its reciprocal. The diagonal matrix Σ is not always full rank so I assume that $ΣΣ^+$ cannot always be an Identity Matrix. How is it possible to prove that the pseudo inverse of $M$, i.e., $M^+$=$VΣ^+U^T$ holds when $ΣΣ^+$ cannot be reduced to Identity Matrix? Is there any other approach?

$M^+=V\Sigma^+U^\ast$ (i.e. $M^+=V\Sigma^+U^\top$ in the real case), where $\Sigma^+$ is a rectangular diagonal matrix whose size is identical to the size of $\Sigma^\top$. The $i$-th main diagonal entry of $\Sigma^+$ is $\sigma_i^{-1}$ if the $i$-th singular value $\sigma_i$ of $M$ is positive, otherwise the diagonal entry is zero.

You may simply prove that $M^+$ is indeed the Moore-Penrose pseudoinverse of $M$ by showing that it satisfies the four defining properties of Moore-Penrose pseudoinverse, namely, both $MM^+$ and $M^+M$ are Hermitian (or real symmetric in your case), $MM^+M=M$ and $M^+MM^+=M^+$.

• While proving those four properties of Moore-Penrose pseudo inverse, we reach a point where we need to reduce $ΣΣ^+$ to $I$. Is there other ways? Oct 30, 2013 at 7:47
• @kungfu Would you please show me why you need to show that $\Sigma\Sigma^+=I$? To prove the four defining properties, you shouldn't need the equality $\Sigma\Sigma^+=I$ and the equality is not true in general. Oct 30, 2013 at 9:08
• Well, if i want to verify that the pseudo inverse follows property 1 of Moore Penrose Pseudo inverse, i.e., $MM^+M$ = $M$ Starting from Left hand side, $MM^+M$ = $(UΣV^T)(VΣ^+U^T)(UΣV^T)$ = $UΣ(V^TV)Σ^+(U^TU)ΣV^T$ = $UΣΣ^+ΣV^T$ How do i move forward here after? Oct 30, 2013 at 10:25
• @kungfu You may prove that $\Sigma\Sigma^+\Sigma=\Sigma$. Suppose $\Sigma$ is $m\times n$ and it has rank $k$. If $m\ge n$, write $\Sigma=\pmatrix{S_{k\times k}&0_{k\times(n-k)}\\ 0_{(n-k)\times k}&0_{(n-k)\times(n-k)}\\ 0_{(m-n)\times k}&0_{(m-n)\times(n-k)}}$ where $S$ is an invertible diagonal matrix. So, you may directly verify that $\Sigma\Sigma^+\Sigma=\Sigma$. The proof is similar if $m<n$. Oct 30, 2013 at 10:37
• Could you please explain your representation of $Σ$? Oct 30, 2013 at 14:03

This is easier to deal with in dyadic (bra-ket) notation.

Write the SVD of $$A$$ as $$A=UDV^\dagger = \sum_k d_k u_k v_k^\dagger,$$ where $$u_k$$ (the left singular vectors) are the columns of $$U$$, $$v_k$$ (the right singular vectors) the columns of $$V$$, and $$d_k\equiv D_{kk}>0$$ the non-zero singular values.

The pseudo-inverse is then simply $$A^+ = \sum_k \frac{1}{d_k} v_k u_k^\dagger.$$ You switch left and right singular vectors and take the reciprocal of the singular values.

You can then also observe directly how $$A^+ A = \sum_k v_k v_k^\dagger = VV^\dagger = \text{projector onto the support of } A, \\ AA^+ = \sum_k u_k u_k^\dagger = UU^\dagger = \text{projector onto the range of }A.$$

(I posted the same answer here).