Pseudoinverse matrix and SVD

I'm trying to solve an homework question but I got stuck.

Let A be a m x n matrix with the SVD $A = U \Sigma V^*$ and $A^+ = (A^* A)^{-1} A^*$ its pseudoinverse.

I'm trying to get $A^+ = V \Sigma^{-1} U^*$, but I'm missing something.

Can anyone help me with this please?

Thanks!

• If you are content with Sivaram Ambikasaran's or my answer you might want to accept one of them so that the question will be recognised as an answered one. – Rasmus Feb 19 '11 at 3:44
• Saying "SVD decomposition" is not quite unlike saying "enter your PIN number into the ATM machine"... – J. M. ain't a mathematician Aug 3 '11 at 8:31
• Fair enough! Thanks for the fix! =) – paulochf Aug 24 '11 at 21:06

\begin{align} A^+ &= (A^*A)^{-1}A^* \\ &=(V\Sigma U^*U\Sigma V^*)^{-1} V\Sigma U^* \\ &=(V\Sigma^2 V^*)^{-1} V\Sigma U^* \\ &=(V^*)^{-1} \Sigma^{-2} V^{-1} V\Sigma U^* \\ &= V \Sigma^{-2}\Sigma U^* \\ &= V\Sigma^{-1}U^* \end{align} using the properties of the matrices $U,V,\Sigma$ in the Singular_value_decomposition.

• Hi Rasmus! I could get by myself until 3rd line. The 4th one was my point of doubt. I forgot to invert the $\left( \cdot \right)^{-1}$ sequence! Thanks in pointing that! =) – paulochf Feb 2 '11 at 15:12
• It could happen that $\Sigma^{-1}$ does not exist. If $\sigma_i=0$ for some $i$. – Herman Jaramillo Aug 6 '19 at 20:44
• If $\Sigma$ is not square (and thus not invertible), the result still holds! Just show directly that $(V \Sigma' \Sigma V')^{-1} = V(\Sigma' \Sigma )^+V'$ and that $(\Sigma' \Sigma )^+ \Sigma' = \Sigma^+$. The result will follow by adjusting the derivation in the answer. I'm not super familiar with the pseudoinverse + notation, but hopefully I'm using it correctly. – Smithey Jun 27 '20 at 2:10

If the dimensions of A are m x n and $m\not\equiv n$ then there isn't any way of deriving $A^+ = U\Sigma^{-1}V$. The reason is because $\Sigma$ has the same dimensions as $A$ therefore it is not invertible. If you see any source about SVD you will see that the equation is $A = U_{mxm} \Sigma_{mxn}V^T_{nxn}$. If A is rectangular maybe the possible derivation you're looking for is \begin{align} A^+ &=(A^TA)^{-1}A^T\\ &=(V\Sigma^TU^T U\Sigma V^T)^{-1}V\Sigma^TU^T\\ &=(V\Sigma^T\Sigma V^T)^{-1}V\Sigma^TU^T \\ &=(V^T)^{-1}(\Sigma^T \Sigma)^{-1}V^{-1}V\Sigma^TU^T \\ &=V(\Sigma^T \Sigma)^{-1}\Sigma^TU^T \end{align}

• The result still holds for non-square matrices (unless I made a mistake); see my comment above – Smithey Jun 27 '20 at 2:12

First you need to assume that the matrix $$A^*A$$ is invertible. For which you need $$n \leq m$$ and rank($$A$$) is $$n$$.

So when $$n \leq m$$ and when rank($$A$$) is $$n$$, then the reduced SVD of $$A$$ is $$A = U \Sigma V^*$$ where $$U \in \mathbb{R}^{m \times n}$$, $$\Sigma \in \mathbb{R}^{n \times n}$$ and $$V \in \mathbb{R}^{n \times n}$$ such that $$U^* U = I_{n \times n}$$, $$V^* V = I_{n \times n}$$, $$V V^* = I_{n \times n}$$ and $$\Sigma$$ is a square diagonal matrix and has only (positive) real entries.

Note that $$V^{-1}=V^*$$.

Also note that $$A^* = V \Sigma^* U^* = V \Sigma U^*$$ since $$\Sigma^* = \Sigma$$.

Further note that if $$M_1,M_2 \text{and} M_3$$ are invertible matrices then $$(M_1 M_2 M_3)^{-1} = M_3^{-1} M_2^{-1} M_1^{-1}$$.

Use these to get the final answer.

• Given that the question is homework - which I realise only now - it would have probably been better to restrict to hints like you did. Sorry. – Rasmus Feb 1 '11 at 22:37
• Hey Sivaram, I was in a hurry when I posted, so it was a (bad one) typo. Thanks for fixing the title! – paulochf Feb 2 '11 at 15:05
• I knew all these hints but I missed your "further note" and did not invert the sequence inside those parentheses. Thanks for your patience! – paulochf Feb 2 '11 at 15:14