Pseudo inverse of a singular value decomposition SVD is equal to its “real” inverse for a square matrix?

Question

I was reading this book on numeric linear algebra and it said pseudo inverse of a singular value decomposition (SVD) is equal to it's "real" inverse for a square matrix. It said it is quite clear that they are equal but I don't really understand how. I know pseudo inverse of a invertible matrix $A* = V\Sigma^{-1} U^T$. But how this is equal to inverse of matrix $A$.

The SVD of a Matrix $A = U\Sigma V^T$, so it's inverse $A^{-1} = (U\Sigma V^T)^{-1}$, how does this equal $V\Sigma^{-1}U^T$?

Maybe I am missing some basic inverse calculation? Can somebody please show this to me?

P.S: This is my first question here, sorry if I made any mistakes

Answer 1

By writing $A^{-1}$ you are assuming the inverse of $A$ exists. Thus, $A$ must be square and $U,\Sigma,V$ must all be square invertible matrices. Hence $A^{-1} = (U\Sigma V^\top)^{-1} = (V^\top)^{-1}\Sigma^{-1}U^{-1}$. Note also that, in singular value decomposition, both $U$ and $V$ are orthogonal matrices and hence $U^{-1}=U^\top$ and $V^{-1}=V^{\top}$. We then get $A^{-1}=V\Sigma^{-1}U^\top$.

Answer 2

Every full rank matrix has a singular value decomposition of the form $$ \mathbf{A} = \mathbf{U} \, \mathbf{S} \, \mathbf{V}^{*} $$ where $\mathbf{S}$ is a diagonal matrix and the domain matrices are unitary: $$ \mathbf{U}\,\mathbf{U}^{*} = \mathbf{U}^{*}\mathbf{U} = \mathbf{I}, \quad \mathbf{V}\,\mathbf{V}^{*} = \mathbf{V}^{*}\mathbf{V} = \mathbf{I}. $$ The Moore-Penrose pseudoinverse matrix is given by $$ \mathbf{A}^{\dagger} = \mathbf{V} \, \mathbf{S}^{-1} \, \mathbf{U}^{*}. $$ Show that for the full rank matrix the pseudoinverse $\mathbf{A}^{\dagger}$ is equivalent to the classic inverse $\mathbf{A}^{-1}$.

We define the classic matrix as having the property that $$ \mathbf{A}^{-1} \mathbf{A} = \mathbf{A}\,\mathbf{A}^{-1} = \mathbf{I}. $$ Show that the pseudoinverse has this property. $$ \begin{align} \mathbf{A}^{-1} \mathbf{A} &= \left( \mathbf{V} \, \mathbf{S}^{-1} \, \mathbf{U}^{*} \right) \left( \mathbf{U} \, \mathbf{S} \, \mathbf{V}^{*} \right) \\ &= \mathbf{V} \, \mathbf{S}^{-1} \underbrace{\left( \mathbf{U}^{*}\mathbf{U}\right)}_{\mathbf{I}} \mathbf{S} \, \mathbf{V}^{*} \\ &= \mathbf{V}\underbrace{\mathbf{S}^{-1}\mathbf{S}}_{\mathbf{I}}\mathbf{V}^{*} \\ &= \mathbf{V}\,\mathbf{V}^{*} \\ &= \mathbf{I} \end{align} $$ Showing $\mathbf{A}\,\mathbf{A}^{\dagger} = \mathbf{I}$ uses the similar machinations.

The presumption of full rank eliminates the nullspaces and provides a full rank, diagonal matrix for $\mathbf{S}$.