Finding the inverse of a matrix using a series I want to find the inverse of the matrix $A$ given by:
$ \left( \begin{array}{cc}
1 & -\epsilon  \\
\epsilon & 1  \\
  \end{array} \right) $
where $|\epsilon|$ $< 1$ (although I do not know how to use this yet)
by finding the matrix $B$ such that $A = I-B$ or $B = I-A$.
$B$ is 
$ \left( \begin{array}{cc}
0 & \epsilon  \\
-\epsilon & 0  \\
  \end{array} \right) $
And I want to find the inverse by summing the series $I + B + B^2 + B^3 . . .$
this series doesn't converge (unless we use the fact about the absolute value given, but I don't know what to do with that here).
 A: Notice that
$$B^2=-\epsilon^2 I$$
so we find easily that
$$B^{2p}=(-1)^p\epsilon^{2p}I$$
and
$$B^{2p+1}=(-1)^p\epsilon^{2p}B$$
hence 
$$\sum_{k=0}^\infty B^k=\sum_{p=0}^\infty B^{2p}+\sum_{p=0}^\infty B^{2p+1}=\left(\sum_{p=0}^\infty (-1)^p\epsilon^{2p}\right)I+\left(\sum_{p=0}^\infty (-1)^p\epsilon^{2p}\right)B\\=\frac{1}{1+\epsilon^2}(I+B)$$
Now since
$$A\sum_{k=0}^\infty B^k=(I-B)\sum_{k=0}^\infty B^k=I$$
then
$$A^{-1}=\frac{1}{1+\epsilon^2}(I+B)$$
A: The basic idea, as demonstrated by Sami Ben Romdhane, is convergence of the series $\sum^\infty_{k=0}B^k$.
It's no coincidence that this kind of series gives rise to an inverse. Remember the geometric series? For $q \in \mathbb{C}$ with $\vert q \vert < 1$ we have that $$\sum^\infty_{k=0}q^k=\frac{1}{1-q}.$$
The operator-theoretic equivalent is the Neumann series: If $B$ is some square matrix and the series $\sum^\infty_{k=0}B^k$ converges (w.r.t. some matrix norm), then we have
$$\sum^\infty_{k=0}B^k = (I - B)^{-1}.$$
Sami Ben Romdhane's answer has shown this convergence and equality by hand, but you could also use the fact that $\Vert B \Vert_{2,2} = \epsilon <1$ (since the eigenvalues of $B$ are $\pm i \epsilon$).
A: Yet another approach, not really based on series expressing $A^{-1}$ as a polynomial in $A$:
If the $n\times n$ matrix $A$ is non-singular, then you can find an expression for $A^{-1}$ in terms of the powers $A^i$, $i=1,\ldots,n-1$, by using the characteristic polynomial $p$ of $A$. Since $A$ is non-singular, the characteristic polynomial 
$$
p(t)=t^n+\alpha_{n-1}t^{n-1}+\cdots+\alpha_2t^2+\alpha_1 t + \alpha_0
$$
is such that $\alpha_0\neq 0$. We know that $p$ annihilates $A$: $p(A)=0$, hence
$$
p(A)=A^{n}+\alpha_{n-1}A^{n-1}+\cdots+\alpha_2t^2+\alpha_1 A+\alpha_0 I=0.
$$
Therefore,
$$
A^{n-1}+\alpha_{n-1}A^{n-2}+\cdots+\alpha_2A+\alpha_1 I+\alpha_0 A^{-1}=0
$$
and
$$
A^{-1}=-\frac{1}{\alpha_0}\left(A^{n-1}+\alpha_{n-1}A^{n-2}+\cdots+\alpha_2A+\alpha_1 I\right).
$$
In your case, $p(t)=t^2-2t+1+\epsilon^2$,    so
$$
A^{-1}=-\frac{1}{1+\epsilon^2}(A-2I).
$$
If $A=I-B$, then 
$$
A^{-1}=\frac{1}{1+\epsilon^2}(I+B).
$$
