Taylor's Polynomial of a Function: how we can arrive at it naturally? Let $f:(a,b)\rightarrow\mathbb{R}$ be a function, which is differentiable, say infinitely many times, in the domain. [We can minimize the hypothesis, but for question, we can assume it.]
Consider the polynomial $P(x)=f(c) + (x-c)f'(c) + \cdots + \frac{f^{k}(c)}{k!}(x-c)^k$.
This polynomial has two properties:
(1) This is a good approximation to $f$ near $c$.
(2) From definition, $P^{(i)}(c)=f^{(i)}(c)$ for $i=1,2,\ldots, k$.
But, I didn't get any "natural motivativation" to arrive at exactly this polynomial.
After discussion with some teachers, they said, it generalizes, mean value theorem: $f(x)-f(x)=f'(\alpha)(x-c)$ i.e. $f(x)=f(c) + f'(\alpha)(x-c)$. But, I didn't get from this mean value theorem, how one naturally arrives at the above $P(x)$, which involves some "factorials", which were not looking in mean value theorem.
In mean value theorem, $f'(\alpha)$ can be written as $f'(\alpha)/1!$ and so one may justify this way for appearance of factorial things; but I would ask then, *we can write $f'(\alpha)$ as $f'(\alpha)/1$', $f'(\alpha)/1^2$' so can't we take the polynomial as  $P^{(i)}(c)=f^{(i)}(c)$ for $i=1,2,\ldots, k$, then can't we take polynomial as $f(c) + (x-c)f'(c) + \cdots + \frac{f^{k}(c)}{k^2}(x-c)^k$?
Why the choice of $P(x)$ above is most natural one and what is "natural way" to arrive at the polynomial $P(x)$?
 A: The usual proof when assuming the most hypothesis on $f$ seems to explain pretty well where the coefficients come from. There are several other proofs if $f$ doesn't satisfy enough hypothesis, giving you different conclusions, but the more hypothesis you assume, the more intuitive is the reasoning. Here is the derivation I'm talking about.
$$\begin{align}f(x)&=f(0)+\int_0^xf'(y)dy\\
&=f(0)+\int_0^x(f'(0)+\int_0^yf''(z)dz)dy\\
&=f(0)+xf'(0)+\int_0^x\int_0^yf''(0)+\int_0^zf'''(w)dwdzdy\\
&=f(0)+xf'(0)+(x^2/2)f''(0)+\int_0^x\int_0^y\int_0^zf'''(w)dwdzdy\\
&=\cdots
\end{align}$$
We see that the coefficient of $f^{(n)}(0)$ is $n$ nested integrals starting by $\int_0^x \cdots dy$ and thus it is the area of a simplex of side $x$ with a right angle. One way to get the coefficients is to use the recursion $\int_0^x y^n/n! dy = x^{n+1}/(n+1)!$.
But there is a lot more to say about this basis $x^n/n!$ that I am not aware of (you find it at a lot of places, not only in Taylor expansion). For instance, a consequence of the things I don't know is that we can think of $x^n/n!$ as "take the product of $n$ copies of $x$ and cancel out the permutations you can make of the factors". Ok, so what I just said probably doesn't make a lot of sense, but one way to get something formal out of it is to say that $x^n$ represents the area of a cube of side $n$
$$ \{(t_1,\dots,t_n)\in\mathbb{R}^n\,|\,0<t_1,\cdots,t_n<x\} $$
while $x^n/n!$ is the area of the simplex
$$ \{(t_1,\dots,t_n)\in\mathbb{R}^n\,|\,0<t_1<\cdots<t_n<x\} $$
since in the cube, we have $n!$ ways to order the coefficients $t_i$ and hence $n!$ copies of the the simplex.
A: I think of Taylor series as a trick that lets you extract coefficients.
Let's say you have a polynomial $p(x) = a_0 + a_1x + a_2x^2 + \cdots + a_n x^n$.  However, the coefficients $a_i$ are unknown; but let's say you're allowed to evaluate $p(x)$ at any point, as well as evaluate the derivative $p^n(x)$ for any $n$, at any point $x$.  For example you can compute $p(3) = 5$, $p'(1) = 0$, $p^3(2) = 4.532$, or whatever.  The values here are just examples; the point is that you can use $p(x)$ and its derivatives as black boxes, but you don't know the formula for $p(x)$. Can you still find the coefficients?
Sure.  Finding $a_0$ is easy: Set $x=0$.  $$a_0 = p(0).$$  Thus, we can always extract the constant coefficient.
To find $a_1$, just note: if we differentiate, the constant term $a_0$ vanishes, and then we can extract the constant again:
$$p'(x) = 0 + a_1 + 2a_2x + 3a_3x^2 + 4a_4x^3 + 5a_5x^4 + \cdots$$
and so $a_1 = p'(0)$.
Continuing to differentiate gives
\begin{align*}
p''(x) &= 2a_2 + 6a_3x + 12a_4x^2 + 20a_5x^3\cdots\\
p'''(x) &= 6a_3 + 24a_4x + 60a_5x^2 + \cdots
\end{align*}
and so $$p''(0) = 2a_2,$$ giving $a_2 = p''(0) / 2$, and similarly $$p'''(0) = 6a_3,$$ so $a_3 = p'''(0) / 6$, and so on.
Hopefully you see the pattern.  The factorials come from the fact that the $n$-th derivative of $x^n$ is the constant $n!$: $$x^n \rightarrow nx^{n-1} \rightarrow n(n-1)x^{n-2} \rightarrow \cdots \rightarrow n! x^0$$ where $\rightarrow$ is differentiation.
This shows that it works for polynomials.  But now if you assume $p(x)$ is a power series $a_0 + a_1x + a_2x^2 + \cdots$, the trick still works.
Now do this a little more carefully, keeping track of the errors in approximating by successive polynomials using some clever integral manipulations, and you get Taylor's Theorem.
