I keep seeing power series throughout mathematics disguised in all different shapes, yet I can't seem to put my finger on what is really fundamentally being expressed here.
Arabic numerals are just a condensed notation for a finite power series: $$ 123 = 3*10^0 + 2*10^1 + 1*10^2 $$
Binary and Hex just replace the 10 with 2 or 16 respectively and adjust some coefficients.
Then there's the Geometric Series, Taylor Series, Fourier Series, Laplace Transform, Z-transform, Moment Generating Functions, Probability Generating Functions, Formal Power Series, Generating Functions.
Some of these are synonymous to varying degrees, some are special cases of one another. But I can't quite bring order into this whole mess.
Edit: Let me elaborate on this question a little:
In the case of the Fourier Series an analogy that is often made is that of a projection onto an orthogonal basis in linear algebra. We create a new world who's atoms are powers of complex exponentials and reconstruct our old object by combining these new atoms into the same shape as before. It's a little like swapping out the periodic table or (a little more realistically) re-encoding a picture in a different file format.
The purpose of this change of reference frame is to hopefully expose patterns that weren't previously visible. An analogy that Douglas Hofstadter makes in Gödel, Escher, Bach is looking at a vineyard from different angles. What seems like chaos from one angle can be highly ordered when viewed from another.
Viewed in this light the question that haunts me with power series is simply: what is it about successively rising powers of x that makes it so natural to use them as a basis for these new spaces? Looking at linear algebra it strikes me as completely obvious why the euclidean basis vectors are so often chosen. But powers of x offer no further intuition to me at all.