# Is there a meaningful pseudo-inverse of a singular projection matrix?

Hello linear algebra experts. In my research I'd like to solve (or approximate) for B, in the form

$A = GBG$

where A and B are symmetric, square matrices and G is a symmetric, square, singular projection matrix ($G^2 = G$). A is also singular. I've done a bit of research on pseudo and generalized inverses. However, since G is a projection matrix, the Moore-Penrose and Drazin inverses trivially imply $G = G^+$. In my case, $G^+AG^+ = B$ is empirically false for these generalized inverses.

Here's a reproducible example coded up in R.

require(Matrix)
B <- matrix(seq(.01, .25, length.out=25), 5)
B <- B + t(B)
diag(B) <- 1
G <- diag(5) - matrix(rep(1,25), 5)/5
A <- G %*% B %*% G
Gpi <- ginv(G)
Gpi %*% A %*% Gpi  # this == A, not B
Gpi %*% Gpi   # == Gpi, not the Identity


From what I understand, the identity matrix is the only projection matrix that is invertible. So the question: is there any pseudo-inverse that can approximate the inverse in this setting? In a least-squares setting perhaps? Or perhaps there's no way to recover the lost 'dimension' by performing the singular projection...

Thanks!

• @what are you trying to solve for? This is unclear to me. Is it B, at given A and G? – Andreas H. Feb 1 '14 at 20:52
• If that is the case, a prerequisite to A is that GAG = A. So it cant be solvable for arbitrary A. And also AG = GA = A – Andreas H. Feb 1 '14 at 20:56
• Sorry, yes, I'm solving for B. Edited original post for clarity. – zzk Feb 1 '14 at 21:13
• Diagonalize $G = Q\Lambda Q^T$, where $\Lambda$ is a diagonal matrix with only ones and zeroes. Then your equation becomes $\tilde A=\Lambda\tilde B\Lambda$, where $\tilde A=Q^TAQ$ and $\tilde B=Q^TBQ$. So $\tilde A$ comes from zeroing out the rows and columns of $\tilde B$ corresponding to the zeroes in the diagonal of $\Lambda$. No way to get them back! You can fill in whatever you like in the missing entries and it will still be a solution... – user856 Feb 1 '14 at 21:35
• Rahul - if the diagonal in $\Lambda$ come from the eigenvalues from $G$, this seems like the same solution you'd get from the Moore-Penrose pseudoinverse of G. $\Lambda$ is still singular, and then you can't solve for $\widetilde{B}$ directly... Or is this your point? – zzk Feb 1 '14 at 22:45

Due to the idempotence property of projections that you quote, a projection is its own unique pseudoinverse, as may be shown by substituting in conditions (I) to (IV) of the definition of a Moore-Penrose pseudoinverse. Other generalised inverses satisfying properties (I) and (II) will in general exist, but the MP pseudoinverse is optimal in a minimum norm sense, that is to say there is nothing better. You already have the least squares solution you are looking for I'm afraid.

Multiplying a matrix by a generalised inverse will not in general give the identity matrix on either side, viz. it may be lacking in rank on both sides, and this is in general the case for projections. The identity matrix is the exception as you observe.

You are correct in suggesting that the lost dimensions cannot be recovered.

I assume, that you want to solve for $B$ at given $A$ and $G$. First we note, that this problem is only solvable, if

$GAG = A$

(evident when we multiply both sides of $A = GBG$ with $G$ and use $G^2=G$).

In that case, $B=A$ is a solution to your problem.

I dont know if that is a solution you had in mind. I guess the approach you are using to solve whatever problem has some fundamental issue.

So, concluding, either your problem is not solvable for given $A$, or, if it is, has the solution $B=A$.

• For my example, B != A, but I can see how no unique solution would exist unless A == B. Now I'm wondering if we remove the equality restriction if there's some G^+ that can minimize |B - G+AG+|. Or maybe some fancier method that can recover after a singular orthogonal projection. Image processing must have to deal with this kind of thing all the time when trying to recover 3d -> 2d image projections. – zzk Feb 1 '14 at 21:23
• Well, I am not saying that $B=A$ is the only solution - it is only one solution. Sure you can ask for a solution in the least squares sense, and that at least exists. But that was not your original question. I think it depends on your application, what the appropriate equation to solve is (that is what I meant by "fundamental issue"). Probably you have not arrived at the "correct" equation yet. – Andreas H. Feb 1 '14 at 21:29
• @zzk Unfortunately, since $G$ is an orthogonal projection, $G = G^+$ is the pseudo inverse that would minimize $\|B-G^+AG^+\|$. In $GBG$, notice that $G$ on the left projects the columns of $BG$ onto the columns of $G$ (and the one on the right means the rows have been projected as well). As Andreas H. points out, you can only have equality if $A$ already has the property that its columns and rows have been projected, in which case $B = G^+AG^+ = GAG = A$. – Travis Bemrose Oct 13 '17 at 17:46
• @zzk $B \ne A \ne GAG$ means $X=GAG$ is the "solution" that minimizes the error $\|A-GXG\|$. It's also the minimum norm "solution" meaning that any other solution with the same amount of error will be of the form $X=GAG+H$ where $GH=HG=0$ and $\| GAG \| \le \| GAG + H\|$. – Travis Bemrose Oct 13 '17 at 20:05