I was trying to prove the following:

Let $U\Sigma V$ be the $SVD$ decomposition of $A\in\mathbb R^{m\times n}$, where $\textrm{rank}(A)=k$. Show that the pseudoinverse of $A$ is given by, $$ \displaystyle A^\dagger=\sum_{i=1}^k\sigma_i^{-1}v_iu_i^T. $$

${\bf Proof:}$ Let us show that $AA^\dagger A=A$, $A^\dagger AA^\dagger=A^\dagger$, $(A^\dagger A)^T=A^\dagger A$ and $(AA^\dagger)^T=AA^\dagger$. Particionate $A$ as follows, \begin{align*} \displaystyle A=\left[\begin{array}{c|c} U_1&U_2 \end{array}\right]\left[\begin{array}{c|c} \tilde{\Sigma}&0\\ \hline 0&0 \end{array}\right]\left[\begin{array}{c} V_1^T\\ V_2^T \end{array}\right], \end{align*} where $\tilde{\Sigma}=\textrm{diag}(\sigma_1, \ldots, \sigma_k)$. Then, $A=U_1\tilde{\Sigma}V_1^T$ whereas $A^\dagger=V_1\tilde{\Sigma}^{-1}U_1^T$. Since $V^TV=U^TU=I$ we see $U_1^TU_1=V_1^TV_1=I$. Then, \begin{align*} \displaystyle (A^\dagger A)^T=(V_1\tilde{\Sigma}U_1^T)(U_1\tilde{\Sigma}^{-1}V_1^T)=I=A^\dagger A. \end{align*} Analogously, we see $(AA^\dagger)=I=AA^\dagger$. Using that $A^\dagger A=AA^\dagger=I$ (how can that be? $A$ is not squared) we have, \begin{align*} \displaystyle (A^\dagger AA^\dagger)=(A^\dagger A)A^\dagger=IA^\dagger\ \textrm{and}\ (AA^\dagger A)=A(A^\dagger A)=AI=A. \end{align*}

${\bf Problems:}$ The main problem is the part $(A^\dagger A)=I$ for $A$ is not a squared matrix. Furthermore if that identity held the problem would be almost trivial.. What is the problem with my proof?


First I guess your SVD decomposition is $A=U\Sigma V^T$ and not $A=U\Sigma V$.

Be carefull of the format of your matrices!

It is true that $V^TV=I_n$ and $U^TU=I_m$ but (unless $n=m$) $I_n\neq I_m$.

Still it is enough to prove that $U_1^T U_1=I_k=V_1^T V_1$.

But then $$\begin{align} A^\dagger\,A & =V_1\tilde \Sigma^{-1} U_1^T U_1\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} I_k\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} \tilde \Sigma V_1^T\\ &=V_1 I_k V_1^T\\ &=V_1 V_1^T\\ &=\left[\begin{array}{c|c} I_k&0\\ \hline 0&0 \end{array}\right] \end{align}\,.$$

And so on.

  • $\begingroup$ Yes it is $A=U\Sigma V^T$.. But then the proof is not fully wrong? If I fix the order problem it gets right? $\endgroup$ – PtF Feb 13 '13 at 12:50
  • $\begingroup$ Well the computation I gave shows that $A^\dagger A\neq I_n$ but $A^\dagger A=\left[\begin{array}{c|c} I_k&0\\ \hline 0&0 \end{array}\right]$ (a $n\times n$ matrix). So you have to modify your proof. But the main idea is good to show that it is a pseudo inverse. $\endgroup$ – Sebastien B Feb 13 '13 at 13:44
  • $\begingroup$ Ok, I will modify.. Thanks for the help.. $\endgroup$ – PtF Feb 13 '13 at 13:58

You shouldn't assume that $A^\dagger A$ is equal to $I$: \begin{align*} A &= U_1 \widetilde{\Sigma} V_1^T,\\ A^\dagger &= V_1 \widetilde{\Sigma}^{-1} U_1^T,\\ \Rightarrow A^\dagger A &= V_1V_1^T \ \text{ is symmetric}. \end{align*}

  • $\begingroup$ But if $A^\dagger A$ is symmetric then $(A^\dagger A)^T=I$ implies $A^\dagger A=I$ but that is not true..I guess the problem was that I was saying $A^\dagger A$ is the "whole" identity $I_{n\times n}$ but its an identity of smaller order as @Sebastien B shows below.. $\endgroup$ – PtF Feb 13 '13 at 12:48
  • $\begingroup$ @Dion I don't understand why you insist that $A^\dagger A=I$. Consider, for instance, $V_1=\begin{pmatrix}1\\0\end{pmatrix}$. Then $V_1V_1^T=\begin{pmatrix}1&0\\0&0\end{pmatrix}$ is symmetric, but it is not equal to $I$. In fact, unless $k=n$, $V_1V_1^T$ is not invertible and hence cannot be equal to $I$. $\endgroup$ – user1551 Feb 13 '13 at 13:04
  • $\begingroup$ I understand now.. Thanks $\endgroup$ – PtF Feb 13 '13 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.