Suppose I have a matrix $$A$$ that is not invertible, but such that $$A + \epsilon I$$ is invertible for all $$\epsilon$$. I'm wondering whether we can say something like $$\underset{\epsilon \to 0}{\lim} (A + \epsilon I)^{-1} = A^{\dagger}$$ where $$A^{\dagger}$$ is the Moore-Penrose pseudoinverse of $$A$$. In essence, can we plug $$\epsilon = 0$$ into the limit, so long as we swap out the inverse for a pseudoinverse? It seems intuitive to me, but I don't see a way to prove or disprove it. Any insight would be appreciated.
• If we define $x_\epsilon$ to be the minimizer for the function $(1/2) \| Ax - b \|_2^2 + \frac{\epsilon}{2} \| x \|_2^2$, then I think we have $\lim_{\epsilon \to 0^+} x_\epsilon = A^\dagger b$. (Is that correct?) This is intuitive because $A^\dagger b$ is the least norm solution to $Ax - \hat b$, where $\hat b$ is the projection of $b$ onto the column space of $A$. May 11 '20 at 6:45
No. Consider the diagonal matrix $$A=\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$$ its pseudo inverse is also $$\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$$.
However $$(A + \epsilon I)^{-1} = \begin{pmatrix} {1\over 1+\epsilon} & 0 \\ 0 & {1\over \epsilon} \end{pmatrix}$$ and the limit is singular.