I understand that they are viewed as approximations, but was that Taylor's original hope?
Assuming that a function can be written as a power series seems to me to be a wild assumption, without some kind of formal argument first being made.
Was there such an argument?
In the best of worlds all functions should be polynomials.
Since this fails, one uses Taylor's formula to replace (some) functions by polynomials which reasonably approximate them.
Mathematicians who can't face these harsh realities become algebraic geometers and live in a dream world where indeed all functions are polynomials.
One possible motivation comes from calculus.
A derivative is nothing more than a local approximation of a functions behaviour by a linear map.
In particular, we have:
$$f(x)=f(a)+f'(x)(x-a)+o(|x-a|) $$
for differentiable functions from $\mathbb{R}^n$ to $\mathbb{R}$.
So a differentiable function is one that can be "well-approximated" by a linear polynomial in the above sense everywhere.
A natural generalisation is to expand our class of approximating functions (linear polynomials) to higher order polynomials in the hope that this looser view will strengthen the bounds we can put on our error term. As it happens this is exactly what happens if we have higher regularity, this is Taylor's theorem.
It is a miracle that for some functions, the Taylor series we construct can not only give us good local approximations by truncating to a finite number of terms, but converges EXACTLY to the original function $f$ in a small neighbourhood of $a$. These are the analytic functions, whose importance becomes especially apparent if one studies complex analysis.
Taylor himself argued for his "Taylor series" on the basis of Newton's forward-difference formula. It may be summarised as follows.
An infinite polynomial $$A+Bx+Cx^2+Dx^3+...$$ has infinite degrees of freedom. Therefore we expect to be able to construct an infinite polynomial passing through an infinite number of given points, just as a parabola of the form $$y=Ax^2+Bx+C$$ can be constructed going through essentially any three points, but not any four, owing to its having three coefficients (similarly you can make $y=Ax+B$ go through any two points etc.).
Newton's forward-difference formula constructs such a polynomial, namely a polynomial which takes the same values as a given function at the x-values $0,b,2b,3b,...$ Taylor's derivation of his series consists in letting $b$ go to zero is this formula.
