Finding pseudoinverse of "non linearly-independent" matrix

If:

1. If $$A$$ has linearly independent columns, $$A^+=\left(A^*A\right)^{-1}A^*$$

2. If $$A$$ has linearly independent rows, $$A^+=A^*\left(AA^*\right)^{-1}$$

3. Otherwise, use the SVD decomposition.

Is it possible to avoid using SVD decomposition?

I've found the following method (https://www.omnicalculator.com/math/pseudoinverse#how-to-calculate-the-pseudoinverse):

1. Start by calculating $$AA^T$$ and row reduce it to reduced row echelon form.
2. Take the non-zero rows of the result and make them the columns of a new matrix $$P$$.
3. Similarly, row-reduce $$A^TA$$ and use its non-zero rows for the columns of the new matrix $$Q$$.
4. With your newly found $$P$$ and $$Q$$, calculate $$M=P^TAQ$$.
5. Finally, calculate the pseudoinverse $$A^+=QM^{-1}P^T$$.

It works fine for most cases, but it doesn't work for $$A=\left [ \begin{matrix} 0&1&0&-i\\0&0&1&0\\0&0&0&0\end{matrix} \right ]$$.

Is the method wrong?

• When you have complex matrices, you tend to want the conjugate transpose instead of the regular transpose. Oct 6, 2022 at 17:37
• And indeed replacing all the transposes with conjugate transposes in the proposed method yields the pseudoinverse of $A$ just fine. Oct 6, 2022 at 17:44
• @Aaron, I got that $M$ is not invertible. Do you have another result. I'm aware of the fact that we need to find the conjugate transpose. Oct 6, 2022 at 18:03
• @eMathHelp You need to use conjugate transposes also when constructing $P$ and $Q$, so the conjugates of the nonzero rows of $AA^*$ and $A^* A$ become the columns of $P$ and $Q$. Oct 6, 2022 at 18:06

The method works just fine for the proposed matrix so long as appropriate care is taken with conjugates. I.e.,

1. Start by calculating $$A A^*$$ and row reduce it.
2. Take the nonzero rows of the result and make their conjugates the columns of $$P$$.
3. Row reduce $$A^* A$$ and use the conjugates of its nonzero rows as the columns of $$Q$$.
4. Let $$M = P^* A Q$$.
5. Then $$A^+ = Q M^{-1} P^*$$.

In particular, with $$A = \begin{bmatrix} 0 & 1 & 0 & -i \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}$$ we get $$A A^* = \begin{bmatrix} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{bmatrix},$$ so we let $$P = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{bmatrix}.$$

Similarly, $$A^* A = \begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & -i \\ 0 & 0 & 1 & 0 \\ 0 & i & 0 & 1 \end{bmatrix},$$ so $$Q = \begin{bmatrix} 0 & 0 \\ 1 & 0 \\ 0 & 1 \\ i & 0 \end{bmatrix}.$$ Note the $$i$$ in the final row, not a $$-i$$, because of us taking the appropriate conjugate.

Then $$M = P^* A Q = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix},$$ which is invertible. If the conjugates were not taken when constructing $$p$$ and $$q$$ (really only $$q$$ in this particular example), the first entry of $$M$$ would instead be $$0$$, which makes it not invertible.

Finally, $$A^+ = Q M^{-1} P^* = \begin{bmatrix} 0 & 0 & 0 \\ 1/2 & 0 & 0 \\ 0 & 1 & 0 \\ i/2 & 0 & 0 \end{bmatrix}$$ which is indeed the pseudoinverse of $$A$$.

Let us briefly review SVD and pseudoinverses so that we can understand why the proceedure you gave actually works, and further we can see it will continue to work if we replace transposes with conjugate transposes when we deal with complex matrices.

We have that $$A^*A$$ and $$AA^*$$ are both positive-semidefinite Hermetian matrices, diagonalizable by the spectral theorem, with the same non-zero eigenvalues. If $$A$$ has SVD $$USV^*$$, then $$AA^*=U(SS^*)U^*$$ and $$A^*A=V(S^*S)V^*$$. Here, $$U, V$$ are unitary matrices of eigenvectors/singular vectors, and if we drop the columns corresponding to singular vectors with singular value $$0$$ and let $$S_0$$ be the square diagonal matrix of the non-zero singular values, we have $$A=USV^*=U_0S_0V_0^*$$. We additionally have $$U_0^*U_0=I, V_0^*V_0=I$$, although not in the other order. While we can define the pseudoinverse in terms of the first SVD, it is slightly easier to use the version where we have dropped the zero singular values and singular vectors: $$A^+=V_0S_0^{-1}U_0^*$$.

If $$B$$ is invertible, then $$\operatorname{RREF}(BM)=\operatorname{RREF}(M)$$. Therefore, $$\operatorname{RREF}(AA^*)=\operatorname{RREF}(U(SS^*)U^*)=\operatorname{RREF}((SS^*)U^*)=\operatorname{RREF}(\delta U^*),$$

where $$\delta$$ is the matrix derived from $$SS^*$$ by replacing the non-zero entries with $$1$$. Since $$\delta U^*$$ is simply zeroing out the rows of $$U^*$$ corresponding to zero singular values, it will be a block matrix whose top is $$U_0^*$$ and whose bottom is zero. Thus, row reducing and removing all the rows of zeros is the same as row reducing $$U_0^*$$. Since row reduction is accomplished by left multiplication by an invertible matrix and since transposition reverses the order or multiplication, we can write $$\operatorname{RREF}U_0^*=XU_0^*$$ for some invertible $$X$$, and thus, $$P=(XU_0^*)^*=U_0X^*$$. Similarly, $$Q=V_0Y^*$$ for some invertible $$Y$$.

The key, now, is that we do not care what $$X$$ and $$Y$$ are as they will cancel out in our construction. If we set $$M=P^*AQ=(XU_0^*)(U_0S_0V_0^*)(V_0Y^*)=XS_0Y^*,$$ then $$QM^{-1}P^*=(V_0Y^*)([Y^*]^{-1}S_0^{-1}X^{-1})(XU_0^*)=V_0S_0^{-1}U_0^*=A^+$$.