# Singular Value Decomposition - Proof

With singular value decomposition we can write the following: $$A = U \Sigma V^{T}$$ $$U^{T}AV=U^{T}U\Sigma V^{T} V$$ Since $U,V$ orthogonal, the above equation leads to the following: $$\Sigma =U^{T}AV$$ I've seen a proof that says the following $$\Sigma^{-1}=V^TA^{-1}U$$

Can someone help with to understand how we ended up to the latter equation.

• $\Sigma$ isn't square in general and thus isn't invertible in general. Even in cases where $\Sigma$ is square (which means that $A$ is square) and $\Sigma$ is invertible, the formula that you've written doesn't give an inverse for $\Sigma$. Dec 29 '16 at 15:06
• Note that if $A$ is square and invertible then $\Sigma^{-1}=V^{T}A^{-1}U$. To see this, use the fact that $U^{T}U=UU^{T}=I$ and $V^{T}V=VV^{T}=I$. Dec 29 '16 at 15:18
• Thanks for pointing out this. It was a mistake which I have corrected now. How do you use the fact you have mentioned? Dec 29 '16 at 16:02
• Usually in the context of the SVD, you use $\Sigma^{-1}$ to denote the matrix $m \times n$ (where $\Sigma$ is $n \times m$) with the diagonal given by the reciprocal of the non-zero entries of $\Sigma$. Dec 29 '16 at 16:03

If all the matrices involved are square and invertible, we have $U^T = U^{-1}$ and $V^{T} = V^{-1}$, so $$\Sigma^{-1} = (U^{-1}AV)^{-1} = V^{-1}A^{-1}U = V^TA^{-1}U$$ as desired.

Singular value decomposition

Begin with a nonzero matrix $$\mathbf{A}\in\mathbb{C}^{m\times n}_{\rho}$$ such that the matrix rank $$\rho and $$\rho. The singular value decomposition, guaranteed to exist, is $$\mathbf{A} = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} = \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right].$$ The SVD provides an orthnormal basis for the four fundamental subspaces: $$\begin{array}{ll} % matrix & subspace \\\hline % \color{blue}{\mathbf{U}_{\mathcal{R}}}\in\mathbb{C}^{m\times\rho} & \color{blue}{\mathcal{R}\left(\mathbf{A}\right)} \\ % \color{blue}{\mathbf{V}_{\mathcal{R}}}\in\mathbb{C}^{n\times\rho} & \color{blue}{\mathcal{R}\left(\mathbf{A}^{*}\right)} \\ % \color{red}{\mathbf{U}_{\mathcal{N}}}\in\mathbb{C}^{m\times m-\rho} & \color{red}{\mathcal{N}\left(\mathbf{A^{*}}\right)} \\ % \color{red}{\mathbf{V}_{\mathcal{N}}}\in\mathbb{C}^{n\times n-\rho} & \color{red}{\mathcal{N}\left(\mathbf{A}\right)} % \end{array}$$ There are $$\rho$$ singular values which are ordered and real: $$\sigma_{1} \ge \sigma_{2} \ge \dots \ge \sigma_{\rho}>0.$$ These singular values for the diagonal matrix of singular values $$\mathbf{S} = \text{diagonal} (\sigma_{1},\sigma_{1},\dots,\sigma_{\rho}) \in\mathbb{R}^{\rho\times\rho}.$$ The $$\mathbf{S}$$ matrix is embedded in the sabot matrix $$\Sigma\in\mathbb{R}^{m\times n}$$ whose shape insures conformability. $$\Sigma = \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] = % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & 0& 0& \dots & 0 \\ 0 & \sigma_{2} &&&&& \\ \vdots && \ddots &&\vdots&& \vdots\\ 0 & & & \sigma_{m} & 0 & \dots & 0 \\\hline 0 & \dots && 0 & 0 & \dots & 0 \\ \vdots & && \vdots & \vdots & \ddots & \vdots \\ 0 & \dots && 0 & 0 & \dots & 0 \\ \end{array} \right]$$

Moore-Penrose pseudoinverse

The components of the SVD can be used to construct the Moore-Penrose pseudoinverse: $$\mathbf{A}^{\dagger} = \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} = \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right].$$ A common source of confusion comes from $$\Sigma$$ manipulations, so the main forms are shown here. Note that $$\mathbf{S}^{\mathrm{T}} = \mathbf{S}$$. Pay close attention to the shapes: \begin{align} % \Sigma_{m\times n} &= \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] , \quad % transpose \Sigma^{\mathrm{T}}_{n\times m} = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] , \quad % inverse \Sigma^{\dagger}_{n\times m} = \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \end{align} This provides a crisp construction of $$\Sigma^{\dagger}$$.

Derivation

To isolate the term $$\Sigma^{\dagger}$$, start with the definition of the psuedoinverse. \begin{align} \mathbf{A}^{\dagger} &= \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\ % \mathbf{V}^{*}\, \mathbf{A}^{\dagger} &= \mathbf{V}^{*}\, \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\ % \mathbf{V}^{*}\mathbf{A}^{\dagger}\, \mathbf{U} &= \Sigma^{\dagger} \, \mathbf{U}^{*}\, \mathbf{U} \\ % \mathbf{V}^{*}\mathbf{A}^{\dagger}\, \mathbf{U} &= \Sigma^{\dagger} % \end{align}