Hint $ $ The inverse can be slickly derived via geometric power series as below (straightforward algebra then proves that it is indeed an inverse).
$$\begin{eqnarray} \rm (1-ba)^{-1} &=&\rm 1+ \color{#0a0}b\color{#c00}a + \color{#0a0}b\color{c00}{ab}\color{#c00}a + \color{#0a0}b\color{0a0}{abab}\color{#c00}a +\,\cdots\\
&=&\rm 1+ \color{#0a0}b\:\! (1\:\! +\, \color{c00}{ab}\ \ +\ \ \color{0a0}{abab}\,\ +\,\cdots\,)\:\!\color{#c00}a\\
&=&\rm 1+ \color{#0a0}b\:\! (1\,-\,ab)^{-1}\color{#c00}a\end{eqnarray}\qquad\qquad$$
Halmos asks why this works in a famous expository article (excerpted below). When I was a student I became interested in this. It turns out that one can give good (and rigorous) explanations. It can be proved that all such rational identities are essentially consequences of geometric power series expansions. For references see this Mathoverflow question. See also Paul Cohn, A remark on the quasi-inverse of a product, Illinois J. Math, 2003. Cohn wrote this paper in reply to my question regarding his viewpoint on this topic.
Geometric series. In a not necessarily commutative ring with
unit (e.g., in the set of all $3 \times 3$ square matrices with real
entries), if $\,1 - ab\,$ is invertible, then $\,1 - ba\,$ is invertible. However
plausible this may seem, few people can see their way
to a proof immediately; the most revealing approach belongs
to a different and distant subject.
Every student knows that
$\,1 - x^2 = (1 + x) (1 - x),\,$
and some even know that
$\,1 - x^3 =(1+x +x^2) (1 - x).\,$
The generalization
$\,1 - x^{n+1} = (1 + x + \cdots + x^n) (1 - x)\,$
is not far away. Divide by $\,1 - x\,$ and let $\,n\,$ tend to infinity;
if $\,|x| < 1,\,$ then $\,x^{n+1}$ tends to $\,0,\,$ and the conclusion is
that
$\frac{1}{1 - x} = 1 + x + x^2 + \cdots.\,$
This simple classical argument begins with easy algebra,
but the meat of the matter is analysis: numbers, absolute
values, inequalities, and convergence are needed not only
for the proof but even for the final equation to make
sense.
In the general ring theory question there are no numbers,
no absolute values, no inequalities, and no limits -
those concepts are totally inappropriate and cannot be
brought to bear. Nevertheless an impressive-sounding
classical phrase, "the principle of permanence of functional
form", comes to the rescue and yields an analytically
inspired proof in pure algebra. The idea is to pretend
that $\,\frac{1}{1 - ba}\,$ can be expanded in a geometric series (which
is utter nonsense), so that
$\,(1 - ba)^{-1} = 1 + ba + baba + bababa + \cdots\,$
It follows (it doesn't really, but it's fun to keep pretending) that
$\,(1 - ba)^{-1} = 1 + b (1 + ab + abab + ababab + \cdots) a.\,$
and, after one more application of the geometric series
pretense, this yields
$\,(1 -ba)^{-1} = 1 + b (1 - ab)^{-1} a.\,$
Now stop the pretense and verify that, despite its unlawful
derivation, the formula works. If, that is, $\, c = (1 - ab)^{-1},\,$
so that $\,(1 - ab)c = c(1 - ab) = 1,\,$ then $\,1 + bca\,$ is the inverse
of $\,1 - ba.\,$ Once the statement is put this way, its
proof becomes a matter of (perfectly legal) mechanical
computation.
Why does it all this work? What goes on here? Why
does it seem that the formula for the sum of an infinite
geometric series is true even for an abstract ring in which
convergence is meaningless? What general truth does
the formula embody? I don't know the answer, but I
note that the formula is applicable in other situations
where it ought not to be, and I wonder whether it deserves
to be called one of the (computational) elements
of mathematics. -- P. R. Halmos [1]
[1] Halmos, P.R. $ $ Does mathematics have elements?
Math. Intelligencer 3 (1980/81), no. 4, 147-153