Finding all left inverses of a matrix

Question

I have to find all left inverses of a matrix

$$A = \begin{bmatrix} 2&-1 \\ 5 & 3\\ -2& 1 \end{bmatrix}$$

I created a matrix to the left of $A$,

$$\begin{bmatrix} a &b &c \\ d &e &f \end{bmatrix} \begin{bmatrix} 2&-1 \\ 5 & 3\\ -2& 1 \end{bmatrix} = \begin{bmatrix} 1&0 \\ 0&1 \end{bmatrix}$$

and I got the following system of equations:

\begin{array} {lcl} 2a+5b-2c & = & 1 \\-a+3b+c & = & 0 \\ 2d+5e-2f & = & 0 \\ -d+3e+f & = & 1 \end{array}

After this step, I am unsure how to continue or form those equations into a solvable matrix, and create a left inverse matrix from the answers of these equations.

Answer 1

Find the left inverse for the rank $\rho = 2$ matrix $$ \mathbf{A} = \left[ \begin{array}{rr} 2 & -1 \\ 5 & 3 \\\ -2 & 1 \end{array} \right] $$ The approach is straightforward. Compute the singular value decomposition $$ \mathbf{A} = \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \tag{1} $$ and use this to construct the Moore-Penrose pseudoinverse matrix $$ \mathbf{A}^{+} = \mathbf{V} \, \Sigma^{+} \, \mathbf{C}^{*} \tag{2} $$

How do we know the pseudoinverse matrix is a left inverse? See generalized inverse of a matrix and convergence for singular matrix, What forms does the Moore-Penrose inverse take under systems with full rank, full column rank, and full row rank?

The singular value decomposition is completed using the recipe for the row space in this post: SVD and the columns — I did this wrong but it seems that it still works, why?

---

---

Compute the SVD

1. Construct product matrix $$ \begin{align} \mathbf{W} &= \mathbf{A}^{*} \mathbf{A}\\ &= % A* \left[ \begin{array}{rcr} 2 & 5 & -1 \\ -1 & 3 & 2 \\ \end{array} \right] % A \left[ \begin{array}{rr} 2 & -1 \\ 5 & 3 \\ -2 & 1 \\ \end{array} \right] \\ % W &= \left[ \begin{array}{cc} 33 & 11 \\ 11 & 11 \\ \end{array} \right] \end{align} $$

---

2. Solve for eigenvalues

The characteristic polynomial is given by $$ p(\lambda) = \lambda^{2} - \lambda \text{ tr }\mathbf{W} + \det \mathbf{W} $$ The trace and determinant are $$ \text{tr }\mathbf{W} = 44, \qquad \det \mathbf{W} = (33-11)11=242 $$ Therefore $$ p(\lambda) = \lambda^{2} - 44 \lambda + 242 $$ The roots are the eigenvalues: $$ \lambda \left( \mathbf{W} \right) = \left\{ 11 \left(2 + \sqrt{2}\right), 11 \left(2-\sqrt{2}\right) \right\} $$ The matrix of singular values is $$ \mathbf{S} = \sqrt{11} \left( \begin{array}{cc} \sqrt{2 + \sqrt{2}} & 0 \\ 0 & \sqrt{2 - \sqrt{2}} \\ \end{array} \right) $$ The sabot matrix is $$ \Sigma = \left[ \begin{array}{c} \mathbf{S} \\ \mathbf{0} \\ \end{array} \right] = \sqrt{11} \left[ \begin{array}{cc} \sqrt{2 + \sqrt{2}} & 0 \\ 0 & \sqrt{2 - \sqrt{2}} \\ 0 & 0 \\ \end{array} \right] $$

---

3. Solve for eigenvectors

First eigenvector

Solving $$ \begin{align} \left( \mathbf{W} - \lambda_{1} \mathbf{I}_{2} \right) w_{1} &= \mathbf{0} \\ % \left[ \begin{array}{cc} 33-11 \left(2+\sqrt{2}\right) & 11 \\ 11 & 11-11 \left(2+\sqrt{2}\right) \\ \end{array} \right] % \left[ \begin{array}{cc} w_{x} \\ w_{y} \\ \end{array} \right] % &= % \left[ \begin{array}{cc} 0 \\ 0 \\ \end{array} \right] % \end{align} $$ produces $$ w_{1} = \left[ \begin{array}{cc} 1+\sqrt{2} \\ 1 \\ \end{array} \right] $$

Second eigenvector

Solving $$ \begin{align} \left( \mathbf{W} - \lambda_{2} \mathbf{I}_{2} \right) w_{2} &= \mathbf{0} \\ % \left[ \begin{array}{cc} 33-11 \left(2-\sqrt{2}\right) & 11 \\ 11 & 11-11 \left(2-\sqrt{2}\right) \\ \end{array} \right] % \left[ \begin{array}{cc} w_{x} \\ w_{y} \\ \end{array} \right] % &= % \left[ \begin{array}{cc} 0 \\ 0 \\ \end{array} \right] % \end{align} $$ produces $$ w_{2} = \left[ \begin{array}{cc} 1 - \sqrt{2} \\ 1 \\ \end{array} \right] $$

---

4. Assemble the domain matrix for the row space $$ \begin{align} \mathbf{V} &= \left[ \begin{array}{cc} \frac{w_{1}}{\lVert w_{1} \rVert} & \frac{w_{2}}{\lVert w_{2} \rVert} \\ \end{array} \right] \\ &= \left[ \begin{array}{cc} % c1 \left( 2 \left(2 + \sqrt{2} \right) \right)^{-\frac{1}{2}} \left[ \begin{array}{c} 1 + \sqrt{2} \\ 1 \end{array} \right] & % c2 \left( 2 \left(2 - \sqrt{2} \right) \right)^{-\frac{1}{2}} \left[ \begin{array}{c} 1 - \sqrt{2} \\ 1 \end{array} \right] \end{array} \right] \end{align} $$

---

5. Compute the domain matrix for the column space

Compute the column vectors for $k=1,2$: $$ \mathbf{U}_{k} = \sigma^{-1}_{k} \mathbf{A} \mathbf{V}_{k} $$

$$ \begin{align} \mathbf{U} &= \left[ \begin{array}{ccc} % c1 \left( \sqrt{22} \left(\sqrt{2}+2\right) \right)^{-1} \left[ \begin{array}{r} 1 + 2\sqrt{2} \\ 8 + 5\sqrt{2} \\ -1 - 2\sqrt{2} \\ \end{array} \right] & % c2 \left( \sqrt{22} \left(\sqrt{2} - 2\right) \right)^{-1} \left[ \begin{array}{r} 1 - 2\sqrt{2} \\ 8 - 5\sqrt{2} \\ -1 + 2\sqrt{2} \\ \end{array} \right] & % c3 \left( 2 \right)^{-\frac{1}{2}} \color{gray}{\left[ \begin{array}{r} 1 \\ 0 \\ -1 \\ \end{array} \right]} % \end{array} \right] \end{align} $$ The third column vector, the gray null space vector, was added by inspection.

---

---

Construct the pseudoinverse

Following the prescription in $(2)$ $$ \mathbf{A}^{+} = \mathbf{V} \, \Sigma^{+} \, \mathbf{U}^{*} = \left[ \begin{array}{rcr} 3 & 2 & -3 \\ -5 & 4 & 5 \\ \end{array} \right] $$

---

---

Verify pseudoinverse

Left inverse

$$ \mathbf{A}^{+}\mathbf{A} = \left[ \begin{array}{rcr} 3 & 2 & -3 \\ -5 & 4 & 5 \\ \end{array} \right] \left[ \begin{array}{rr} 2 & -1 \\ 5 & 3 \\ -2 & 1 \\ \end{array} \right] = \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ \end{array} \right] = \mathbf{I}_{2} $$ Therefore, $\mathbf{A}^{+}$ is a left inverse.

$$ \mathbf{A} \mathbf{A}^{+} = \left[ \begin{array}{rr} 2 & -1 \\ 5 & 3 \\ -2 & 1 \\ \end{array} \right] \left[ \begin{array}{rcr} 3 & 2 & -3 \\ -5 & 4 & 5 \\ \end{array} \right] = \frac{1}{2} \left[ \begin{array}{rcr} 1 & 0 & -1 \\ 0 & 2 & 0 \\ -1 & 0 & 1 \\ \end{array} \right] \ne \mathbf{I}_{3} $$ Therefore, $\mathbf{A}^{+}$ is not a right inverse.

Answer 2

Hint:

solve the first two equations using $c$ as a parameter and find $b=\frac1{11}$ , $a=c+\frac3{11}$

do the same in last two equation and you have the result.

Answer 3

Okay. I got the following matrix as my final answer as the general form of all left inverses of A:

\begin{bmatrix} a & \frac1{11} &a-\frac3{11} \\ d & \frac2{11} &d+\frac5{11} \end{bmatrix}

Answer 4

Alternative less advanced solution. Multiply it with its own transpose:

$A^TA$ is a $2\times2$ matrix:

$\left( \begin{array}{cc} 33 & 11 \\ 11 & 11 \\ \end{array} \right)$

This has a trivial inverse by swapping the diagonals and finding the determinant:

$\frac{1}{22}\left( \begin{array}{cc} 1 & -1 \\ -1 & 3 \\ \end{array} \right)$

If you multiply this with the original matrix:

$(A^TA)^{-1}A^TA=I_2$

And note that this is your inverse:

$(A^TA)^{-1}A^T$

Which gives:

$\begin{pmatrix}\frac{3}{22}&\frac{1}{11}&-\frac{3}{22}\\ -\frac{5}{22}&\frac{2}{11}&\frac{5}{22}\end{pmatrix}$

---

Even less advanced, your system of linear equations work too. What I think you're not realising is, due to the very fact that there are less equations than variables, that there are an infinite number of inverses to matrices. Due to that fact you can take the liberty of just letting two variables be 0, and solving the rest. If you choose $c=f=0$, you can quite easily solve $b=\frac{1}{11}, a=\frac{3}{11}, e=\frac{2}{11}, d=-\frac{5}{11}$, which gives the solution:

$\begin{pmatrix} \frac{3}{11}&\frac{1}{11}&0\\ -\frac{5}{11}&\frac{2}{11}&0 \end{pmatrix}$

Whilst this method is superior from a linear algebra perspective (because unlike the first solution above I can find multiple by choosing different values for a pair of unknowns), it's less practical if all you want is to find another inverse.

---

A third idea I have is to use block matrices. Generally speaking I can write your matrix as a joining of the vector $\mathbf{a}=\begin{pmatrix}-2&1\end{pmatrix}$ with the matrix $\begin{pmatrix} 2&-1 \\ 5 & 3 \end{pmatrix}$

Written as a block matrix:

\begin{bmatrix} \mathbf{a}\\M_a \end{bmatrix} Writing the inverse, $B$, as a block matrix of a similar form (albeit with a vetrical vector), we get:

$\begin{bmatrix} \mathbf{b}&M_b \end{bmatrix} \begin{bmatrix} \mathbf{a}\\M_a \end{bmatrix} =I_2$

By the property of block matrices, this gives us: $I_2=\mathbb{a}\mathbb{b}+M_aM_b$

Hence for convenience, if $\mathbf{b}$ is a $0$ vector (the equivalent of setting $a=d=0$ in your linear equations), then all you need to do is invert $M_a$:

$M_b=\begin{pmatrix} 5&3\\ -2&1 \end{pmatrix}^{-1}=\begin{pmatrix}\frac{1}{11}&-\frac{3}{11}\\ \frac{2}{11}&\frac{5}{11}\end{pmatrix}$

Hence your answer would be:

$B=\begin{pmatrix}0&\frac{1}{11}&-\frac{3}{11}\\ 0& \frac{2}{11}&\frac{5}{11}\end{pmatrix}$

For explanatory purposes I wrote everything out. In general, just invert the largest square you can fit (in any position) in the matrix you wish to invert, and just bung it in a $0$ matrix with reversed dimensions (in the same position along the height or width)