What is wrong with my proof: Pseudoinverse and SVD

Question

I was trying to prove the following:

Let $U\Sigma V$ be the $SVD$ decomposition of $A\in\mathbb R^{m\times n}$, where $\textrm{rank}(A)=k$. Show that the pseudoinverse of $A$ is given by, $$ \displaystyle A^\dagger=\sum_{i=1}^k\sigma_i^{-1}v_iu_i^T. $$

${\bf Proof:}$ Let us show that $AA^\dagger A=A$, $A^\dagger AA^\dagger=A^\dagger$, $(A^\dagger A)^T=A^\dagger A$ and $(AA^\dagger)^T=AA^\dagger$. Particionate $A$ as follows, \begin{align*} \displaystyle A=\left[\begin{array}{c|c} U_1&U_2 \end{array}\right]\left[\begin{array}{c|c} \tilde{\Sigma}&0\\ \hline 0&0 \end{array}\right]\left[\begin{array}{c} V_1^T\\ V_2^T \end{array}\right], \end{align*} where $\tilde{\Sigma}=\textrm{diag}(\sigma_1, \ldots, \sigma_k)$. Then, $A=U_1\tilde{\Sigma}V_1^T$ whereas $A^\dagger=V_1\tilde{\Sigma}^{-1}U_1^T$. Since $V^TV=U^TU=I$ we see $U_1^TU_1=V_1^TV_1=I$. Then, \begin{align*} \displaystyle (A^\dagger A)^T=(V_1\tilde{\Sigma}U_1^T)(U_1\tilde{\Sigma}^{-1}V_1^T)=I=A^\dagger A. \end{align*} Analogously, we see $(AA^\dagger)=I=AA^\dagger$. Using that $A^\dagger A=AA^\dagger=I$ (how can that be? $A$ is not squared) we have, \begin{align*} \displaystyle (A^\dagger AA^\dagger)=(A^\dagger A)A^\dagger=IA^\dagger\ \textrm{and}\ (AA^\dagger A)=A(A^\dagger A)=AI=A. \end{align*}

${\bf Problems:}$ The main problem is the part $(A^\dagger A)=I$ for $A$ is not a squared matrix. Furthermore if that identity held the problem would be almost trivial.. What is the problem with my proof?

Answer 1

First I guess your SVD decomposition is $A=U\Sigma V^T$ and not $A=U\Sigma V$.

Be carefull of the format of your matrices!

It is true that $V^TV=I_n$ and $U^TU=I_m$ but (unless $n=m$) $I_n\neq I_m$.

Still it is enough to prove that $U_1^T U_1=I_k=V_1^T V_1$.

But then $$\begin{align} A^\dagger\,A & =V_1\tilde \Sigma^{-1} U_1^T U_1\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} I_k\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} \tilde \Sigma V_1^T\\ &=V_1 I_k V_1^T\\ &=V_1 V_1^T\\ &=\left[\begin{array}{c|c} I_k&0\\ \hline 0&0 \end{array}\right] \end{align}\,.$$

And so on.

Answer 2

You shouldn't assume that $A^\dagger A$ is equal to $I$: \begin{align*} A &= U_1 \widetilde{\Sigma} V_1^T,\\ A^\dagger &= V_1 \widetilde{\Sigma}^{-1} U_1^T,\\ \Rightarrow A^\dagger A &= V_1V_1^T \ \text{ is symmetric}. \end{align*}