2
$\begingroup$

With singular value decomposition we can write the following: \begin{equation} A = U \Sigma V^{T} \end{equation} \begin{equation} U^{T}AV=U^{T}U\Sigma V^{T} V \end{equation} Since $U,V$ orthogonal, the above equation leads to the following: \begin{equation} \Sigma =U^{T}AV \end{equation} I've seen a proof that says the following \begin{equation} \Sigma^{-1}=V^TA^{-1}U \end{equation}

Can someone help with to understand how we ended up to the latter equation.

$\endgroup$
4
  • $\begingroup$ $\Sigma$ isn't square in general and thus isn't invertible in general. Even in cases where $\Sigma$ is square (which means that $A$ is square) and $\Sigma$ is invertible, the formula that you've written doesn't give an inverse for $\Sigma$. $\endgroup$ Dec 29 '16 at 15:06
  • $\begingroup$ Note that if $A$ is square and invertible then $\Sigma^{-1}=V^{T}A^{-1}U$. To see this, use the fact that $U^{T}U=UU^{T}=I$ and $V^{T}V=VV^{T}=I$. $\endgroup$ Dec 29 '16 at 15:18
  • $\begingroup$ Thanks for pointing out this. It was a mistake which I have corrected now. How do you use the fact you have mentioned? $\endgroup$
    – user105627
    Dec 29 '16 at 16:02
  • $\begingroup$ Usually in the context of the SVD, you use $\Sigma^{-1}$ to denote the matrix $m \times n$ (where $\Sigma$ is $n \times m $) with the diagonal given by the reciprocal of the non-zero entries of $\Sigma$. $\endgroup$
    – Batman
    Dec 29 '16 at 16:03
1
$\begingroup$

If all the matrices involved are square and invertible, we have $U^T = U^{-1}$ and $V^{T} = V^{-1}$, so $$ \Sigma^{-1} = (U^{-1}AV)^{-1} = V^{-1}A^{-1}U = V^TA^{-1}U $$ as desired.

$\endgroup$
1
$\begingroup$

Singular value decomposition

Begin with a nonzero matrix $\mathbf{A}\in\mathbb{C}^{m\times n}_{\rho}$ such that the matrix rank $\rho<m$ and $\rho<n$. The singular value decomposition, guaranteed to exist, is $$ \mathbf{A} = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} = \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right]. $$ The SVD provides an orthnormal basis for the four fundamental subspaces: $$ \begin{array}{ll} % matrix & subspace \\\hline % \color{blue}{\mathbf{U}_{\mathcal{R}}}\in\mathbb{C}^{m\times\rho} & \color{blue}{\mathcal{R}\left(\mathbf{A}\right)} \\ % \color{blue}{\mathbf{V}_{\mathcal{R}}}\in\mathbb{C}^{n\times\rho} & \color{blue}{\mathcal{R}\left(\mathbf{A}^{*}\right)} \\ % \color{red}{\mathbf{U}_{\mathcal{N}}}\in\mathbb{C}^{m\times m-\rho} & \color{red}{\mathcal{N}\left(\mathbf{A^{*}}\right)} \\ % \color{red}{\mathbf{V}_{\mathcal{N}}}\in\mathbb{C}^{n\times n-\rho} & \color{red}{\mathcal{N}\left(\mathbf{A}\right)} % \end{array} $$ There are $\rho$ singular values which are ordered and real: $$ \sigma_{1} \ge \sigma_{2} \ge \dots \ge \sigma_{\rho}>0. $$ These singular values for the diagonal matrix of singular values $$ \mathbf{S} = \text{diagonal} (\sigma_{1},\sigma_{1},\dots,\sigma_{\rho}) \in\mathbb{R}^{\rho\times\rho}. $$ The $\mathbf{S}$ matrix is embedded in the sabot matrix $\Sigma\in\mathbb{R}^{m\times n}$ whose shape insures conformability. $$ \Sigma = \left[ \begin{array}{c} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] = % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & 0& 0& \dots & 0 \\ 0 & \sigma_{2} &&&&& \\ \vdots && \ddots &&\vdots&& \vdots\\ 0 & & & \sigma_{m} & 0 & \dots & 0 \\\hline 0 & \dots && 0 & 0 & \dots & 0 \\ \vdots & && \vdots & \vdots & \ddots & \vdots \\ 0 & \dots && 0 & 0 & \dots & 0 \\ \end{array} \right] $$

Moore-Penrose pseudoinverse

The components of the SVD can be used to construct the Moore-Penrose pseudoinverse: $$ \mathbf{A}^{\dagger} = \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} = \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right]. $$ A common source of confusion comes from $\Sigma$ manipulations, so the main forms are shown here. Note that $\mathbf{S}^{\mathrm{T}} = \mathbf{S}$. Pay close attention to the shapes: $$ \begin{align} % \Sigma_{m\times n} &= \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] , \quad % transpose \Sigma^{\mathrm{T}}_{n\times m} = \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] , \quad % inverse \Sigma^{\dagger}_{n\times m} = \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % \end{align} $$ This provides a crisp construction of $\Sigma^{\dagger}$.

Derivation

To isolate the term $\Sigma^{\dagger}$, start with the definition of the psuedoinverse. $$ \begin{align} \mathbf{A}^{\dagger} &= \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\ % \mathbf{V}^{*}\, \mathbf{A}^{\dagger} &= \mathbf{V}^{*}\, \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\ % \mathbf{V}^{*}\mathbf{A}^{\dagger}\, \mathbf{U} &= \Sigma^{\dagger} \, \mathbf{U}^{*}\, \mathbf{U} \\ % \mathbf{V}^{*}\mathbf{A}^{\dagger}\, \mathbf{U} &= \Sigma^{\dagger} % \end{align} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.