One possible motivation comes from calculus.
A derivative is nothing more than a local approximation of a functions behaviour by a linear map.
In particular, we have:
$$f(x)=f(a)+f'(x)(x-a)+o(|x-a|) $$
for differentiable functions from $\mathbb{R}^n$ to $\mathbb{R}$.
So a differentiable function is one that can be "well-approximated" by a linear polynomial in the above sense everywhere.
A natural generalisation is to expand our class of approximating functions (linear polynomials) to higher order polynomials in the hope that this looser view will strengthen the bounds we can put on our error term. As it happens this is exactly what happens if we have higher regularity, this is Taylor's theorem.
It is a miracle that for some functions, the Taylor series we construct can not only give us good local approximations by truncating to a finite number of terms, but converges EXACTLY to the original function $f$ in a small neighbourhood of $a$. These are the analytic functions, whose importance becomes especially apparent if one studies complex analysis.