# Limit of matrix inverse: $\lim_{\lambda \to \infty} (A + \lambda I)^{-1} = \mathbf{0}$?

Let matrix $$A \in \mathbb{R}^{n\times n}$$ be positive semidefinite.

• Is it then true to that $$(A + \lambda I)^{-1} \to \mathbf{0} \quad (\lambda \to \infty) \quad ?$$

• If so, is the fact that $$A$$ is positive definite irrelevant here?

My thoughts so far: $$(A + \lambda I)^{-1} = \Big(\lambda( \frac{1}{\lambda}A + I ) \Big)^{-1} = \frac{1}{\lambda} \Big(\frac{1}{\lambda}A + I \Big)^{-1}$$ I think that $$\lim_{\lambda \to \infty} \Big( \frac{1}{\lambda}A + I \Big)^{-1} = I^{-1} = I$$, but I don't know if I can just pass the $$\lim$$ through the inverse $$(\cdot)^{-1}$$ like that. If this is the case, then $$\lim_{\lambda \to \infty} (A + \lambda I)^{-1} = \lim_{\lambda \to \infty} (1/\lambda) \lim_{\lambda \to \infty} (A/\lambda + I)^{-1} = 0 \cdot I = \mathbf{0}$$ as I'd like to show.

Where this comes from:

I'm trying to justify a claim made in an econometrics lecture. Namely,

$$\textrm{Var}(\hat{\beta}^{\textrm{ridge}}) = \sigma^2 (X^{T}X + \lambda I)^{-1} X^T X [(X^T X + \lambda I)^{-1}]^T \to \mathbf{0}$$ where $$\hat{\beta}^\textrm{ridge}$$ is the ridge estimator in a linear model, $$X \in \mathbb{R}^{n \times p}$$ is the design matrix, and the equality is known. The limit, however, wasn't justified.

• $A$ can be any matrix above. The point is, the inverse of a matrix is a continuous function in a neighbourhood of the identity, therefore since $A - \lambda I$ is going to eventually be invertible, we may pass the limit inside the inverse by continuity, giving the desired result by the continuity of scalar multiplication. Jan 24, 2019 at 17:27
• If $\|\cdot\|$ is a matrix norm, then the Neumann series guarantees that $A+\lambda I$ is invertible with $$(A+\lambda I)^{-1} = \sum_{n=0}^{\infty} \frac{(-1)^n}{\lambda^{n+1}}A^n,$$ which converges uniformly on the region $|\lambda| \geq \|A\|+\delta$ for any given $\delta > 0$. By the Weierstrass M-test, the limit as $\lambda\to\infty$ can be evaluated term-wise, proving the desired claim. Jan 24, 2019 at 17:34
• @астонвіллаолофмэллбэрг Great! That completes my line of reasoning. For others looking on, here's why there is a neighborhood of $I$ in $M_n(\mathbb{R})$ in which $(\cdot)^{-1}$ is continuous: $(\cdot)^{-1} : GL_n(\mathbb{R}) \to GL_n(\mathbb{R})$ is continuous and $GL_n(\mathbb{R})$ is open in $M_n(\mathbb{R})$ (see: math.stackexchange.com/a/810675/369800). [To understand the proof just linked: determinant continuous (see: math.stackexchange.com/a/121834/369800) and adjoint continuous (see: math.stackexchange.com/a/2031642/369800)] Jan 24, 2019 at 19:07
• Recall that the inverse matrix is the adjugate matrix divided by the discriminant. Thus a "singularity" of the inversion only happens when the discriminant vanishes. Jan 26, 2019 at 20:45
• @Mah I would like to think we can do so, but the argument is likely to be more convoluted. I think because $B^TB$ is positive definite, we can lower bound the smallest eigenvalue of $A+\lambda B^TB$ so that it goes to infinity with $\lambda$, then we can be done. Feb 22, 2021 at 4:09

The eigenvalues of $$A+\lambda I$$ are of the form $$\lambda+\mu$$, where $$\mu$$ is an eigenvalue of $$A$$ (necessarily real). Then, for $$\lambda$$ sufficiently large, the eigenvalues of $$A+\lambda I$$ are all $$>1$$.

Note that a matrix $$S$$ that diagonalizes $$A$$ also diagonalizes $$A+\lambda I$$, let $$A=SDS^{-1}$$, with $$D$$ diagonal.

Then $$(A+\lambda I)^{-1}$$ is diagonalizable with eigenvalues in $$(0,1)$$ and therefore $$\lim_{\lambda\to\infty}(A+\lambda I)^{-1}= S\Bigl(\,\lim_{\lambda\to\infty}(D+\lambda I)^{-1}\Bigr)S^{-1}=0$$ It is not necessary that $$A$$ is semipositive definite. Any symmetric matrix will do.

The answer I liked the best was left in the comments by астон вілла олоф мэллбэрг, since it shows that $$A$$ does not need any special structure. Here I'm pulling his answer down and including a bit more detail.

We have $$(A + \lambda I)^{-1} = \Big(\lambda( \frac{1}{\lambda}A + I ) \Big)^{-1} = \frac{1}{\lambda} \Big(\frac{1}{\lambda}A + I \Big)^{-1},$$ and we claim that $$\Big(\frac{1}{\lambda}A + I \Big)^{-1} \to I^{-1} = I \quad (\lambda \to \infty)$$. Therefore, $$(A + \lambda I)^{-1} = \frac{1}{\lambda} \Big(\frac{1}{\lambda}A + I \Big)^{-1} \to 0 \cdot I = \mathbf{0} \quad (\lambda \to \infty),$$ which was the desired result.

We complete the proof by showing the claim. Since $$GL_n(\mathbb{R})$$ is open in $$M_n(\mathbb{R})$$, we find some $$\epsilon > 0$$ such that the open ball $$B(I, \epsilon) \subseteq GL_n(\mathbb{R})$$. Hence, for sufficiently large $$\lambda$$, we know that $$(A/\lambda + I) \in B(I, \epsilon) \subseteq GL_n(\mathbb{R})$$. Also knowing that $$(\cdot)^{-1} : GL_n \to GL_n$$ is continuous, we have $$\lim_{\lambda \to \infty}\Big(\frac{1}{\lambda}A + I \Big)^{-1} = \Big(\lim_{\lambda \to \infty} \frac{1}{\lambda}A + I \Big)^{-1}= (I)^{-1} = I,$$ which completes the proof.

To understand the linked proof of the continuity of $$(\cdot)^{-1}$$, see here for justification that the determinant operator is continuous and here for justification that the adjoint operator is continuous.

• I may not be астон вілла олоф мэллбэрг anymore, but Teresa is very pleased that her idea was useful, even if it was two years ago! Feb 22, 2021 at 4:49