Does $\frac{1}{1-x} = 1+x+x^2+\cdots$ work for certain matrices? In real analysis, there's a theorem saying that for all real $x$ with $|x| <1$, we have: $$\frac{1}{1-x} = 1+x+x^2+\cdots$$

Question. Does a version of this apply to certain matrices, thereby giving us a convenient way of approximating $(1-A)^{-1}$? If so, for which matrices is this formula correct?

 A: If the series $\sum_{n\ge 0} A^n$ converges, then its sum is $(I-A)^{-1} $ since $(I-A)(1+ A+ \cdot + A^{n}) = I - A^{n+1} \to I$. 
Now, we have the following equivalent statements:


*

*The series $\sum_{n\ge 0} A^n $ is convergent

*$A^{n} \to 0$ as $n\to \infty$

*All of the eigenvalues of $A$ have absolute value $< 1$.


1.$\implies$ 2. is clear, let's check 2.$\implies$ 1. Assume $A^n\to 0$. Then for $n$ large enough the matrix $(I-A^{n+1}$ is invertible, and so $I- A$ also is ( from the above). Therefore, $\sum_{k=0}^n A^k = (I-A^{n+1})(I-A)^{-1} \to (I-A)^{-1}$. 


*$\implies$ 3. follows from the fact that if $\{\lambda_i\}$ are the eigenvalues of $A$ then $\{\lambda_i^n\}$ are the eigenvalues of $A^n$

*$\implies 2$. Use the Jordan canonical form, it's enough to check for a Jordan cell $J_{\lambda, d}$ with $|\lambda|< 1$. 
As an observation, if for a certain submultiplicative norm $||\cdot ||$  on the algebra of matrices we have $||A||< 1$ then $A^n \to 0$, but the converse is not true. 
A: Yes, this works for matrices A with $r(A)<1$ and is known as Neumann series. The proof is the same as for numbers. Here $r(A)$ is the spectral radius of $A$:
$$r(A)= \lim_{n \to \infty}\|A^n\|^{1/n}.$$
($\|\cdot\| $ is a submultiplictive norm)
In this case we have that $I-A$ is invertible and
$$(I-A)^{-1}=\sum_{n=0}^{\infty}A^n.$$
A: Yes, this works for matrices $A$ with $\|A\|<1$ and is known as Neumann series. The proof is basically the same as for numbers…
A: As long as the series on the right-hand side converges, so that it's well-defined, you can perform the multiplication
$$
(I-A)(I+A+A^2+\cdots)
$$and see that you do indeed end up with $I$, which means that the two are inverses of one another.
One can, of course, make this result strict with $\epsilon$-$\delta$ and partial sums. It shouldn't be more difficult than most $\epsilon$-$\delta$ proofs.
