Upper bounds on the approximate inverse of a singular matrix

Let $A$ be a singular matrix and $\zeta>0$ such that $A+\zeta I$ is nonsingular.

Soft question: Is it true that if $\zeta$ is sufficiently small, then $$(A+\zeta I)^{-1}A \approx I$$ (or $A(A+\zeta I)^{-1} \approx I$)?

More precisely: Let $E:=(A+\zeta I)^{-1}A - I$. If the answer to the above (soft) question is "yes", what are some known bounds on $E$?

Note. Using the condition number $\kappa(A)$, a well-known bound is $$\frac{\Vert(A+B)^{-1} - A^{-1}\Vert}{\Vert A^{-1}\Vert} \le \kappa(A)\frac{\Vert B\Vert}{\Vert A\Vert },$$

but this bound requires that both $A$ and $A+B$ are nonsingular. In my question, $A$ is singular while $A+B$ is nonsingular.

Try some trivial test cases, like $$A=\pmatrix{1&0\\0&0}\implies A(A+\zeta I)^{-1}=\pmatrix{(1+ζ)^{-1}&0\\0&0}$$ which is nowhere close to the identity matrix.

• I wonder if it generally converges to the projection on the range, as it does for your example. I suspect that this generally holds when $A$ is symmetric, at least. – Omnomnomnom Jun 25 '18 at 14:52
• @Omnomnomnom can you elaborate on this? – JohnA Jun 25 '18 at 14:59
• @JohnA would a result that applies only to symmetric matrices be useful? – Omnomnomnom Jun 25 '18 at 16:35
• @Omnomnomnom yes, but I would also be interested in how you are connecting this question with the range projection of $A$ more generally (even if there isn't a precise theorem or proof available). – JohnA Jun 25 '18 at 16:42

There must exist some bound on $E$. But it's not true that $||E||\to0$ as $\zeta\to0$, which would seem to say that the answer to the slightly fuzzy question of whether $(A+\zeta I)^{-1}A \approx I$ is no:

There exists $x\ne0$ with $Ax=0$. Hence $Ex=-x$, so $||E||\ge1$.

When $A$ is diagonalizable, we can find a change of basis $S$ such that $M = S^{-1}AS$ is diagonal, and moreover $$M = \pmatrix{D & 0\\0 &0}$$ where $D$ is a diagonal and invertible matrix. We see that $$M(M + \zeta I)^{-1} = \pmatrix{D(D + \zeta I)^{-1} & 0\\0 &0}.$$ As $\zeta \to 0$, we see that $M(M + \zeta I)^{-1}$ approaches the projection onto the range of $M$ along the kernel of $M$. Since we merely applied a change of basis, we can conclude that $A(A + \zeta I)^{-1}$ approaches the projection $P$ onto the range of $A$ along the kernel of $A$.

That is, $A(A + \zeta I)^{-1} \to P$ where $P$ satisfies:

• $P^2 = P$
• $PAx = Ax$ for all $x$
• $Px = 0 \iff Ax = 0$

In general, $A(A + \zeta I)^{-1}$ approaches the projection onto the image of $A^n$ along the kernel of $A^n$, where $n$ is the size of the matrix. This holds whether or not $A$ is diagonalizable.

• What is meant by "projection onto the range of A along the kernel of A"? – JohnA Jun 25 '18 at 17:16
• That’s what the explanation about $P$ explains – Omnomnomnom Jun 25 '18 at 17:19
• @Omnomnomnom Hmmm it seems $(A+\zeta I)^{-1}\to A^+$, the pseudo-inverse of $A$... – AccidentalFourierTransform Jun 25 '18 at 18:44
• @AccidentalFourierTransform my first thought was something involving the pseudo inverse as well. However, $AA^+$ is an orthogonal projection, which is not generally true for the limit of $A(A + \zeta I)^{-1}$, so we can rule that out. I think your idea holds for symmetric matrices, though – Omnomnomnom Jun 25 '18 at 19:31
• @AccidentalFourierTransform $(A^*A+\zeta I)^{-1}A^* \to A^+$; see arxiv.org/pdf/1110.6882.pdf – JohnA Jul 4 '18 at 17:59