5
$\begingroup$

I'm trying to solve an homework question but I got stuck.

Let A be a m x n matrix with the SVD $A = U \Sigma V^*$ and $A^+ = (A^* A)^{-1} A^*$ its pseudoinverse.

I'm trying to get $A^+ = V \Sigma^{-1} U^*$, but I'm missing something.

Can anyone help me with this please?

Thanks!

$\endgroup$
3
  • $\begingroup$ If you are content with Sivaram Ambikasaran's or my answer you might want to accept one of them so that the question will be recognised as an answered one. $\endgroup$ – Rasmus Feb 19 '11 at 3:44
  • 1
    $\begingroup$ Saying "SVD decomposition" is not quite unlike saying "enter your PIN number into the ATM machine"... $\endgroup$ – J. M. isn't a mathematician Aug 3 '11 at 8:31
  • $\begingroup$ Fair enough! Thanks for the fix! =) $\endgroup$ – paulochf Aug 24 '11 at 21:06
9
$\begingroup$

$$ \begin{align} A^+ &= (A^*A)^{-1}A^* \\ &=(V\Sigma U^*U\Sigma V^*)^{-1} V\Sigma U^* \\ &=(V\Sigma^2 V^*)^{-1} V\Sigma U^* \\ &=(V^*)^{-1} \Sigma^{-2} V^{-1} V\Sigma U^* \\ &= V \Sigma^{-2}\Sigma U^* \\ &= V\Sigma^{-1}U^* \end{align} $$ using the properties of the matrices $U,V,\Sigma$ in the Singular_value_decomposition.

$\endgroup$
3
  • $\begingroup$ Hi Rasmus! I could get by myself until 3rd line. The 4th one was my point of doubt. I forgot to invert the $\left( \cdot \right)^{-1}$ sequence! Thanks in pointing that! =) $\endgroup$ – paulochf Feb 2 '11 at 15:12
  • $\begingroup$ It could happen that $\Sigma^{-1}$ does not exist. If $\sigma_i=0$ for some $i$. $\endgroup$ – Herman Jaramillo Aug 6 '19 at 20:44
  • $\begingroup$ If $\Sigma$ is not square (and thus not invertible), the result still holds! Just show directly that $(V \Sigma' \Sigma V')^{-1} = V(\Sigma' \Sigma )^+V'$ and that $(\Sigma' \Sigma )^+ \Sigma' = \Sigma^+$. The result will follow by adjusting the derivation in the answer. I'm not super familiar with the pseudoinverse + notation, but hopefully I'm using it correctly. $\endgroup$ – Smithey Jun 27 '20 at 2:10
3
$\begingroup$

First you need to assume that the matrix $A^*A$ is invertible. For which you need $n \leq m$ and rank($A$) is $n$.

So when $n \leq m$ and when rank($A$) is $n$, then the reduced SVD of $A$ is $A = U \Sigma V^*$ where $U \in \mathbb{R}^{m \times n}$, $\Sigma \in \mathbb{R}^{n \times n}$ and $V \in \mathbb{R}^{n \times n}$ such that $U^* U = I_{n \times n}$, $V^* V = I_{n \times n}$, $V V^* = I_{n \times n}$ and $\Sigma$ is a square diagonal matrix and has only (positive) real entries.

Note that $V^{-1}=V^*$.

Also note that $A^* = V \Sigma^* U^* = V \Sigma U^*$ since $\Sigma^* = \Sigma$.

Further note that if $M_1,M_2 \text{and} M_3$ are invertible matrices then $(M_1 M_2 M_3)^{-1} = M_3^{-1} M_2^{-1} M_1^{-1}$.

Use these to get the final answer.

$\endgroup$
3
  • $\begingroup$ Given that the question is homework - which I realise only now - it would have probably been better to restrict to hints like you did. Sorry. $\endgroup$ – Rasmus Feb 1 '11 at 22:37
  • $\begingroup$ Hey Sivaram, I was in a hurry when I posted, so it was a (bad one) typo. Thanks for fixing the title! $\endgroup$ – paulochf Feb 2 '11 at 15:05
  • $\begingroup$ I knew all these hints but I missed your "further note" and did not invert the sequence inside those parentheses. Thanks for your patience! $\endgroup$ – paulochf Feb 2 '11 at 15:14
2
$\begingroup$

If the dimensions of A are m x n and $m\not\equiv n$ then there isn't any way of deriving $A^+ = U\Sigma^{-1}V$. The reason is because $\Sigma$ has the same dimensions as $A$ therefore it is not invertible. If you see any source about SVD you will see that the equation is $A = U_{mxm} \Sigma_{mxn}V^T_{nxn}$. If A is rectangular maybe the possible derivation you're looking for is $$\begin{align} A^+ &=(A^TA)^{-1}A^T\\ &=(V\Sigma^TU^T U\Sigma V^T)^{-1}V\Sigma^TU^T\\ &=(V\Sigma^T\Sigma V^T)^{-1}V\Sigma^TU^T \\ &=(V^T)^{-1}(\Sigma^T \Sigma)^{-1}V^{-1}V\Sigma^TU^T \\ &=V(\Sigma^T \Sigma)^{-1}\Sigma^TU^T \end{align}$$

$\endgroup$
1
  • $\begingroup$ The result still holds for non-square matrices (unless I made a mistake); see my comment above $\endgroup$ – Smithey Jun 27 '20 at 2:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.