Why the null space of pseudo inverse equals the null space of the matrix transpose?

The pseudoinverse $A^+$ of A is the matrix for which $x = A^+Ax$ for all x in the row space of A. The nullspace of $A^+$ is the nullspace of $A^T$.

I don't understand this cause the above seems to imply that $A^+=A^T$ which doesn't make sense as $x = A^+Ax$ while $A^TA$ gives a matrix which is not an identity matrix.

Here is the source-Page 2 on "Finding the pseudo Inverse"

• Use the SVD of $A$ to express the pseudoinverse of $A$. The relationship should then become clear. – K. Miller Nov 24 '16 at 13:27
• I will get $A^+=VΣ^+U^T$ where matrix $Σ^+$ whose first r rows have $\frac{1}{σ_1}, \frac{1}{σ_2}, ..., \frac{1}{σ_r}$ on the diagonal. But I feel to see why or the intuition behind how that will equate to the transpose. – newbie125 Nov 24 '16 at 14:04
• Well $A^T = V\Sigma^TU^T$. They both have the same nullspace, namely, $U_{r+1:m}$, where $A$ is an $m\times n$ matrix. – K. Miller Nov 24 '16 at 16:37
• So the Σ doesn't matter but it is because both consists of the matrix multiplication between V and $U^T$ right? The Σ is more like the scalar product between the two right? – newbie125 Nov 24 '16 at 17:47

Define matrix

Start with a matrix $$\mathbf{A} \in\mathbb{C}^{m\times n}_{\rho}$$

Fundamental Theorem of Linear Algebra

The Fundamental Theorem of Linear Algebra can be expressed as \begin{align} % \mathbf{C}^{n} = \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \oplus \color{red}{\mathcal{N} \left( \mathbf{A} \right)} \\ % \mathbf{C}^{m} = \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \oplus \color{red} {\mathcal{N} \left( \mathbf{A}^{*} \right)} % \end{align}

Singular value decomposition

The singular value decomposition of the matrix is \begin{align} \mathbf{A} &= \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\ % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & & & \dots & 0 \\ 0 & \sigma_{2} \\ \vdots && \ddots \\ & & & \sigma_{\rho} \\\hline & & & & 0 & \\ \vdots &&&&&\ddots \\ 0 & & & & & & 0 \\ \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % & = % U \left[ \begin{array}{cccccccc} \color{blue}{u_{1}} & \dots & \color{blue}{u_{\rho}} & \color{red}{u_{\rho+1}} & \dots & \color{red}{u_{n}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{v_{1}^{*}} \\ \vdots \\ \color{blue}{v_{\rho}^{*}} \\ \color{red}{v_{\rho+1}^{*}} \\ \vdots \\ \color{red}{v_{n}^{*}} \end{array} \right] % \end{align} The connection to the Fundamental Theorem is intimate: $$\begin{array}{ll} % column \ vectors & span \\\hline % \color{blue}{u_{1}} \dots \color{blue}{u_{\rho}} & \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \\ % \color{blue}{v_{1}} \dots \color{blue}{v_{\rho}} & \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \\ % \color{red}{u_{\rho+1}} \dots \color{red}{u_{m}} & \color{red}{\mathcal{N} \left( \mathbf{A}^{*} \right)} \\ % \color{red}{v_{\rho+1}} \dots \color{red}{v_{n}} & \color{red}{\mathcal{N} \left( \mathbf{A} \right)} \\ % \end{array}$$

Pseudoinverse matrix

The least squares solution with the SVD produces the pseudoinverse:

\begin{align} \mathbf{A}^{+} &= \mathbf{V} \, \Sigma^{+} \mathbf{U}^{*} \\ % &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % % U \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} % $$\mathbf{A}^{+} \in\mathbb{C}^{n\times m}_{\rho}$$

The subspace decomposition in terms of the pseudoinverse is now explicit.

For comparison, the adjoint matrix is \begin{align} \mathbf{A}^{*} &= \mathbf{V} \, \Sigma^{T} \mathbf{U}^{*} \\ % &= % V \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % % U \left[ \begin{array}{c} \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{U}_{\mathcal{N}}}^{*} \end{array} \right] \\ % \end{align} $$\mathbf{A}^{*} \in\mathbb{C}^{n\times m}_{\rho}$$