3
$\begingroup$

For a matrix $M$ with its singular value decomposition $UΣV^T$, the pseudo inverse of $M$, i.e., $M^+$ is $VΣ^+U^T$.

  1. How can I derive the pseudo inverse(Moore–Penrose) $M^+$ from the singular value decomposition of a matrix $M$?
  2. From SVD, we know that $Σ$ is a diagonal matrix which contains the square roots of the eigen values of both $MM^T$ and $M^TM$ whereas $Σ^+$ is formed by replacing the non-zero diagonal elements of $Σ$ by its reciprocal. The diagonal matrix Σ is not always full rank so I assume that $ΣΣ^+$ cannot always be an Identity Matrix. How is it possible to prove that the pseudo inverse of $M$, i.e., $M^+$=$VΣ^+U^T$ holds when $ΣΣ^+$ cannot be reduced to Identity Matrix? Is there any other approach?
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

$M^+=V\Sigma^+U^\ast$ (i.e. $M^+=V\Sigma^+U^\top$ in the real case), where $\Sigma^+$ is a rectangular diagonal matrix whose size is identical to the size of $\Sigma^\top$. The $i$-th main diagonal entry of $\Sigma^+$ is $\sigma_i^{-1}$ if the $i$-th singular value $\sigma_i$ of $M$ is positive, otherwise the diagonal entry is zero.

You may simply prove that $M^+$ is indeed the Moore-Penrose pseudoinverse of $M$ by showing that it satisfies the four defining properties of Moore-Penrose pseudoinverse, namely, both $MM^+$ and $M^+M$ are Hermitian (or real symmetric in your case), $MM^+M=M$ and $M^+MM^+=M^+$.

$\endgroup$
6
  • $\begingroup$ While proving those four properties of Moore-Penrose pseudo inverse, we reach a point where we need to reduce $ΣΣ^+$ to $I$. Is there other ways? $\endgroup$
    – kungfu
    Oct 30, 2013 at 7:47
  • $\begingroup$ @kungfu Would you please show me why you need to show that $\Sigma\Sigma^+=I$? To prove the four defining properties, you shouldn't need the equality $\Sigma\Sigma^+=I$ and the equality is not true in general. $\endgroup$
    – user1551
    Oct 30, 2013 at 9:08
  • $\begingroup$ Well, if i want to verify that the pseudo inverse follows property 1 of Moore Penrose Pseudo inverse, i.e., $MM^+M$ = $M$ Starting from Left hand side, $MM^+M$ = $(UΣV^T)(VΣ^+U^T)(UΣV^T)$ = $UΣ(V^TV)Σ^+(U^TU)ΣV^T$ = $UΣΣ^+ΣV^T$ How do i move forward here after? $\endgroup$
    – kungfu
    Oct 30, 2013 at 10:25
  • $\begingroup$ @kungfu You may prove that $\Sigma\Sigma^+\Sigma=\Sigma$. Suppose $\Sigma$ is $m\times n$ and it has rank $k$. If $m\ge n$, write $\Sigma=\pmatrix{S_{k\times k}&0_{k\times(n-k)}\\ 0_{(n-k)\times k}&0_{(n-k)\times(n-k)}\\ 0_{(m-n)\times k}&0_{(m-n)\times(n-k)}}$ where $S$ is an invertible diagonal matrix. So, you may directly verify that $\Sigma\Sigma^+\Sigma=\Sigma$. The proof is similar if $m<n$. $\endgroup$
    – user1551
    Oct 30, 2013 at 10:37
  • $\begingroup$ Could you please explain your representation of $Σ$? $\endgroup$
    – kungfu
    Oct 30, 2013 at 14:03
1
$\begingroup$

This is easier to deal with in dyadic (bra-ket) notation.

Write the SVD of $A$ as $$A=UDV^\dagger = \sum_k d_k u_k v_k^\dagger,$$ where $u_k$ (the left singular vectors) are the columns of $U$, $v_k$ (the right singular vectors) the columns of $V$, and $d_k\equiv D_{kk}>0$ the non-zero singular values.

The pseudo-inverse is then simply $$A^+ = \sum_k \frac{1}{d_k} v_k u_k^\dagger.$$ You switch left and right singular vectors and take the reciprocal of the singular values.

You can then also observe directly how $$A^+ A = \sum_k v_k v_k^\dagger = VV^\dagger = \text{projector onto the support of } A, \\ AA^+ = \sum_k u_k u_k^\dagger = UU^\dagger = \text{projector onto the range of }A.$$

(I posted the same answer here).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.