# Can I find a power series representation for any $x$ in the domain just by varying the center?

It is well known that $\displaystyle\dfrac {1}{1-x} = \sum_{n=0}^{\infty}(x-0)^n$ when $x \in (-1, 1)$. If I want a power series representation on the interval $(2, 4)$, can I simply find a Taylor series for $f(x) = \dfrac {1}{1-x}$ centered at $3$? Can I continue moving the center around until I cover $\mathbb{R}$ \ $\{-1, 1\}$? If not, why would we ever use a center that is not $0$?

Yes. Observe that: $\frac{1}{1-x}=\frac{-1}{2}\frac{1}{1+(x-3)/2}=\frac{-1}{2}\sum_{n=0}(-1)^{n}\frac{1}{2^n}(x-3)^n$, as long as $x\in(1,5)$ since the radius of convergence of the power series is 2. In fact, for any $a\in\mathbb{R}-\{1\}$, You can write $1/(1-x)$ as a power series with center at $x=a$, and the power series converges to $1/(1-x)$ as long as $x\in(a-|a-1|,a+|a-1|)$.
The sum $$\sum_{n=1}^{M} -x^{-n} = \frac{1/x-(1/x)^{M+1}}{1/x-1} = \frac{1}{1-x} - \frac{x^{-M}}{1-x}$$ converges to $\frac{1}{1-x}$ for $\lvert x \rvert > 1$. This is akin to a Taylor series at $\infty$, but is called a Laurent series because it contains powers of $1/x$.
One can find the Taylor series of $\frac{1}{1-x}$ about the point $x=a$ by writing it as $$\frac{1}{1-a-(x-a)} = \frac{1}{1-a} \frac{1}{1-(x-a)/(1-a)} = \sum_{n=0}^{\infty} \frac{(x-a)^n}{(1-a)^{n+1}}$$ by again using the geometric series formula (Clearly $a=0$ recovers the original series about $x=0$). This series converges if $$\left| \frac{x-a}{1-a} \right| < 1.$$ You therefore find, for example, if $a=3$, the series converges if $-2<x-3<2$, i.e. $1<x<5$. This will work for any $a \neq 1$. The general principle is that the series cannot converge on an interval that includes the singularity at $x=1$. In this case, this is the only obstacle to convergence.
The usefulness of the series expansion generally lies in the ability to approximate the function as closely as we like over a finite interval by only taking a finite number of terms (if you happen to know that $12^2=144$, you can get a pretty good estimate for $\sqrt{145}$, to take a very simple example). Hence expanding about a point closer to points you are interested in provides a better estimate for less cost than a further away point (such as trying to estimate $\sqrt{145}$ based on the series expansion of $\sqrt{x}$ at $100$ instead of $144$: it's pretty obvious you won't get the same level of accuracy without taking more terms).