# Why the null space of pseudo inverse equals the null space of the matrix transpose?

The pseudoinverse $A^+$ of A is the matrix for which $x = A^+Ax$ for all x in the row space of A. The nullspace of $A^+$ is the nullspace of $A^T$.

I don't understand this cause the above seems to imply that $A^+=A^T$ which doesn't make sense as $x = A^+Ax$ while $A^TA$ gives a matrix which is not an identity matrix.

Here is the source-Page 2 on "Finding the pseudo Inverse"

• Use the SVD of $A$ to express the pseudoinverse of $A$. The relationship should then become clear. Nov 24, 2016 at 13:27
• I will get $A^+=VΣ^+U^T$ where matrix $Σ^+$ whose first r rows have $\frac{1}{σ_1}, \frac{1}{σ_2}, ..., \frac{1}{σ_r}$ on the diagonal. But I feel to see why or the intuition behind how that will equate to the transpose. Nov 24, 2016 at 14:04
• Well $A^T = V\Sigma^TU^T$. They both have the same nullspace, namely, $U_{r+1:m}$, where $A$ is an $m\times n$ matrix. Nov 24, 2016 at 16:37
• So the Σ doesn't matter but it is because both consists of the matrix multiplication between V and $U^T$ right? The Σ is more like the scalar product between the two right? Nov 24, 2016 at 17:47

## Define matrix

Start with a matrix $$\mathbf{A} \in\mathbb{C}^{m\times n}_{\rho}$$

## Fundamental Theorem of Linear Algebra

The Fundamental Theorem of Linear Algebra can be expressed as \begin{align} % \mathbf{C}^{n} = \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \oplus \color{red}{\mathcal{N} \left( \mathbf{A} \right)} \\ % \mathbf{C}^{m} = \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \oplus \color{red} {\mathcal{N} \left( \mathbf{A}^{*} \right)} % \end{align}

## Singular value decomposition

The singular value decomposition of the matrix is \begin{align} \mathbf{A} &= \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\ % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & & & \dots & 0 \\ 0 & \sigma_{2} \\ \vdots && \ddots \\ & & & \sigma_{\rho} \\\hline & & & & 0 & \\ \vdots &&&&&\ddots \\ 0 & & & & & & 0 \\ \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % & = % U \left[ \begin{array}{cccccccc} \color{blue}{u_{1}} & \dots & \color{blue}{u_{\rho}} & \color{red}{u_{\rho+1}} & \dots & \color{red}{u_{n}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{v_{1}^{*}} \\ \vdots \\ \color{blue}{v_{\rho}^{*}} \\ \color{red}{v_{\rho+1}^{*}} \\ \vdots \\ \color{red}{v_{n}^{*}} \end{array} \right] % \end{align} The connection to the Fundamental Theorem is intimate: $$\begin{array}{ll} % column \ vectors & span \\\hline % \color{blue}{u_{1}} \dots \color{blue}{u_{\rho}} & \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \\ % \color{blue}{v_{1}} \dots \color{blue}{v_{\rho}} & \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \\ % \color{red}{u_{\rho+1}} \dots \color{red}{u_{m}} & \color{red}{\mathcal{N} \left( \mathbf{A}^{*} \right)} \\ % \color{red}{v_{\rho+1}} \dots \color{red}{v_{n}} & \color{red}{\mathcal{N} \left( \mathbf{A} \right)} \\ % \end{array}$$

## Pseudoinverse matrix

The least squares solution with the SVD produces the pseudoinverse:

\begin{align} \mathbf{A}^{+} &= \mathbf{V} \, \Sigma^{+} \mathbf{U}^{*} \\ % &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % % U \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} % $$\mathbf{A}^{+} \in\mathbb{C}^{n\times m}_{\rho}$$

The subspace decomposition in terms of the pseudoinverse is now explicit.

For comparison, the adjoint matrix is \begin{align} \mathbf{A}^{*} &= \mathbf{V} \, \Sigma^{T} \mathbf{U}^{*} \\ % &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % % U \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} $$\mathbf{A}^{*} \in\mathbb{C}^{n\times m}_{\rho}$$

• What does "⊕" indicate? And what exactly is $V_R$ vs $V_N$? Mar 15, 2023 at 4:24
• The symbol ⊕ represents a [direct sum] (en.wikipedia.org/wiki/Direct_sum#Examples) between range space elements and null space elements, spaces that only share the point at the origin. The row space $\mathbf{V}$ resolves into range space $\color{blue}{\mathbf{V}_{\mathcal{R}}}$ and null space $\color{red}{\mathbf{V}_{\mathcal{N}}}$ components. Mar 18, 2023 at 17:56