Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I was trying to prove the following:

Let $U\Sigma V$ be the $SVD$ decomposition of $A\in\mathbb R^{m\times n}$, where $\textrm{rank}(A)=k$. Show that the pseudoinverse of $A$ is given by, $$ \displaystyle A^\dagger=\sum_{i=1}^k\sigma_i^{-1}v_iu_i^T. $$

${\bf Proof:}$ Let us show that $AA^\dagger A=A$, $A^\dagger AA^\dagger=A^\dagger$, $(A^\dagger A)^T=A^\dagger A$ and $(AA^\dagger)^T=AA^\dagger$. Particionate $A$ as follows, \begin{align*} \displaystyle A=\left[\begin{array}{c|c} U_1&U_2 \end{array}\right]\left[\begin{array}{c|c} \tilde{\Sigma}&0\\ \hline 0&0 \end{array}\right]\left[\begin{array}{c} V_1^T\\ V_2^T \end{array}\right], \end{align*} where $\tilde{\Sigma}=\textrm{diag}(\sigma_1, \ldots, \sigma_k)$. Then, $A=U_1\tilde{\Sigma}V_1^T$ whereas $A^\dagger=V_1\tilde{\Sigma}^{-1}U_1^T$. Since $V^TV=U^TU=I$ we see $U_1^TU_1=V_1^TV_1=I$. Then, \begin{align*} \displaystyle (A^\dagger A)^T=(V_1\tilde{\Sigma}U_1^T)(U_1\tilde{\Sigma}^{-1}V_1^T)=I=A^\dagger A. \end{align*} Analogously, we see $(AA^\dagger)=I=AA^\dagger$. Using that $A^\dagger A=AA^\dagger=I$ (how can that be? $A$ is not squared) we have, \begin{align*} \displaystyle (A^\dagger AA^\dagger)=(A^\dagger A)A^\dagger=IA^\dagger\ \textrm{and}\ (AA^\dagger A)=A(A^\dagger A)=AI=A. \end{align*}

${\bf Problems:}$ The main problem is the part $(A^\dagger A)=I$ for $A$ is not a squared matrix. Furthermore if that identity held the problem would be almost trivial.. What is the problem with my proof?

share|cite|improve this question
up vote 1 down vote accepted

First I guess your SVD decomposition is $A=U\Sigma V^T$ and not $A=U\Sigma V$.

Be carefull of the format of your matrices!

It is true that $V^TV=I_n$ and $U^TU=I_m$ but (unless $n=m$) $I_n\neq I_m$.

Still it is enough to prove that $U_1^T U_1=I_k=V_1^T V_1$.

But then $$\begin{align} A^\dagger\,A & =V_1\tilde \Sigma^{-1} U_1^T U_1\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} I_k\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} \tilde \Sigma V_1^T\\ &=V_1 I_k V_1^T\\ &=V_1 V_1^T\\ &=\left[\begin{array}{c|c} I_k&0\\ \hline 0&0 \end{array}\right] \end{align}\,.$$

And so on.

share|cite|improve this answer
Yes it is $A=U\Sigma V^T$.. But then the proof is not fully wrong? If I fix the order problem it gets right? – PtF Feb 13 '13 at 12:50
Well the computation I gave shows that $A^\dagger A\neq I_n$ but $A^\dagger A=\left[\begin{array}{c|c} I_k&0\\ \hline 0&0 \end{array}\right]$ (a $n\times n$ matrix). So you have to modify your proof. But the main idea is good to show that it is a pseudo inverse. – Sebastien B Feb 13 '13 at 13:44
Ok, I will modify.. Thanks for the help.. – PtF Feb 13 '13 at 13:58

You shouldn't assume that $A^\dagger A$ is equal to $I$: \begin{align*} A &= U_1 \widetilde{\Sigma} V_1^T,\\ A^\dagger &= V_1 \widetilde{\Sigma}^{-1} U_1^T,\\ \Rightarrow A^\dagger A &= V_1V_1^T \ \text{ is symmetric}. \end{align*}

share|cite|improve this answer
But if $A^\dagger A$ is symmetric then $(A^\dagger A)^T=I$ implies $A^\dagger A=I$ but that is not true..I guess the problem was that I was saying $A^\dagger A$ is the "whole" identity $I_{n\times n}$ but its an identity of smaller order as @Sebastien B shows below.. – PtF Feb 13 '13 at 12:48
@Dion I don't understand why you insist that $A^\dagger A=I$. Consider, for instance, $V_1=\begin{pmatrix}1\\0\end{pmatrix}$. Then $V_1V_1^T=\begin{pmatrix}1&0\\0&0\end{pmatrix}$ is symmetric, but it is not equal to $I$. In fact, unless $k=n$, $V_1V_1^T$ is not invertible and hence cannot be equal to $I$. – user1551 Feb 13 '13 at 13:04
I understand now.. Thanks – PtF Feb 13 '13 at 13:58

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.