Query about the Moore Penrose pseudoinverse method

Question

I have recently discovered the Moore-Penrose psuedoinverse method, and I am currently testing the waters with it. I noticed if I have a system, say

$$a_1x_1=0$$ $$a_2x_1+a_3x_2=0$$ $$\vdots$$ $$a_5x_1+a_6x_2=0$$

we can write it in matrix form as $$AX=\hat{0}$$ where $$A=\begin{bmatrix}a_1 & 0\\ a_2 & a_3 \\ \vdots & \vdots\\ a_5& a_6 \end{bmatrix},\qquad X= [x_1\;\; x_2],\quad \hat{0}=\begin{bmatrix}0\\ \vdots\\ 0\end{bmatrix}$$

after doing all the necessary calculations it does indeed find an (left) inverse. My question is upon observation if $a_1\neq 0 $ then clearly $x_1=0$ is a solution and this would fix $x_2=0$, but the solutions obtained via the Moore-Penrose pseudoinverse method doesn't give these trivial solutions. Would the left inverses obtained via this method indeed be the correct solutions to this system?

Answer 1

There is confusion in either my understanding or the question. Start with a full column rank matrix, $$ \mathbf{A} \in \mathbb{R}^{m\times 2}_{2} $$ with $m>2$. The span of the row space is the entire plane: $$ \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} = \mathbb{R}^{2}. $$ The colors distinguish $\color{blue}{range}$ spaces from $\color{red}{null}$ space.

Next, you construct the Moore-Penrose pseudoinverse, $\mathbf{A}^{\dagger}$. Recall this pseudoinverse is a projector onto the range space $\color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)}$.

Problems:

1. It seems you are looking for a $\color{red}{null}$ space solution with a $\color{blue}{range}$ space tool.
2. There is no $\color{red}{null}$ space $\color{red}{\mathcal{N} \left( \mathbf{A} \right)}$ for this problem.

Perhaps this toy problem will help. $$ \begin{align} \mathbf{A} x & = b \\ % \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \\ \end{array} \right] % \left[ \begin{array}{c} x_{1} \\ x_{2} \\ \end{array} \right] % &= % \left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] % \end{align} $$ The Moore-Penrose pseudoinverse is $$ \mathbf{A}^{\dagger} = % \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \\ \end{array} \right]. $$

Case 1: No existence

$$\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] =\left[ \begin{array}{c} 0 \\ b_{2} \\ \end{array} \right]$$

The data vector is in the null space: $b\in\color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)}$. There is no solution.

Case 2: Existence and uniqueness

$$\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] =\left[ \begin{array}{c} b_{1} \\ 0 \\ \end{array} \right]$$

The exclusion is $b_{1}\ne0$. There is no $\color{red}{null}$ space: $\color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)} = \mathbf{0}$. The data vector is in the $\color{blue}{range}$ space: $b\in\color{blue}{\mathcal{R}\left( \mathbf{A}\right)}$. The solution exists and is unique. The direct solution is also the least squares solution $$ x = x_{LS} = \left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] $$

Case 3: Existence, no uniqueness

$$\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] =\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right]$$

he exclusion is $b_{1}\ne0$, $b_{2}\ne0$. Now there is a $\color{red}{null}$ space: $\color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)}$. The data vector inhabits both $\color{blue}{range}$ and $\color{red}{null}$ space: $$ b = \color{blue}{b_{\mathcal{R}}} + \color{red}{b_{\mathcal{N}}} $$ A solution exists and it is not unique. There is no direct solution. The least square solution includes an arbitrary vector $y\in\mathbb{R}^{2}$: $$ x_{LS} = \color{blue}{\mathbf{A}^{\dagger}b} + \color{red}{\left( \mathbf{I}_{2} - \mathbf{A}^{\dagger}\mathbf{A}\right)^{-1} y} = \color{blue}{ \left[ \begin{array}{c} b_{1} \\ 0 \\ \end{array} \right]} + \alpha \color{red}{ \left[ \begin{array}{c} 0 \\ 1 \\ \end{array} \right]} $$