Understanding the Taylor Series

Question

I'll admit I've always had a few confusions about the Taylor Series, as it itself looks completely unintuitive to me.

$$f(x) = f(x_0)+f'(x_0)(x-x_0)+\frac{f''(x_0)(x-x_0)^2}{2!}+\frac{f'''(x_0)(x-x_0)^3}{3!}+ \ ...$$

This looks really weird to me for a couple reasons (just the general expansion of $f(x)$):

• Why is a point $x_0$ required to expand the function?

• Why is the term $(x-x_0)$ and not $(x+x_0)$? I know shifting a graph to the right involves taking $f(x)$ to $f(x-x_0)$ but I've never really understood why intuitively, and why this is needed to expand the series.

Taking the derivative of $f(x) = f'(x)$.

$$\therefore f'(x) = \frac{d}{dx}\left(f(x_0)+f'(x_0)(x-x_0)+\frac{f''(x_0)(x-x_0)^2}{2!}+\frac{f'''(x_0)(x-x_0)^3}{3!}+ \ ...\right)$$ $$f'(x) = f'(x_0)+ f''(x_0)\frac{d}{dx}[(x-x_0)]+ f'''(x_0)/2!\frac{d}{dx}[(x-x_0)]^2 \ + \ ...$$

$$f'(x) = f'(x_0) - f''(x_0)+f'''(x_0)(x-x_0)\ + \ ...$$

Now there are two constant terms, and the term to first have the $(x-x_0)$ multiplied to it is now the third term. I don't see any intuitive pattern here or anything.

I suppose what I'm asking for is an explanation for the intuition going on here other than the expansion matches any function evaluated at a point for every derivative. I'm asking for the two bulleted points to be addressed, with an explanation. I've tried reading about it a bit now and it hasn't been coming nicely to me, but if someone instead thinks I should read about it somewhere they think is good with explaining it I welcome that as a comment too.

Answer 1

One way to think of Taylor series is as a generalization of the tangent line to a curve. At the point $x_0$ the tangent line to the graph of $y = f(x)$ can be written as

$$y = f(x_0) + f'(x_0)(x - x_0)$$

which is also called the linearization of $f$ at $x_0$: approximating the function by a linear/degree $1$ polynomial. If you want to approximate the function with a tangent "parabola" add higher-order derivatives and powers:

$$y = f(x_0) + f'(x_0)(x - x_0) + \dfrac{f''(x_0)}{2!}(x - x_0)^2$$

approximates the function with a degree $2$ polynomial at $x_0$.

If you want to approximate the function with a tangent "cubic" then continue the pattern:

$$y = f(x_0) + f'(x_0)(x - x_0) + \dfrac{f''(x_0)}{2!}(x - x_0)^2 + \dfrac{f'''(x_0)}{3!}(x - x_0)^3$$

Taylor series allow you to continue this to a degree $n$ polynomial:

$$y = f(x_0) + f'(x_0)(x - x_0) + \dfrac{f''(x_0)}{2!}(x - x_0)^2 + \dots +\dfrac{f^{(n)}(x_0)}{n!}(x - x_0)^n$$

Answer 2

There exists a polynomial $p(x)$ such that all of its derivatives at some point ($x_0$), will equal all of the derivatives of $f(x)$ at $x_0$

The Taylor formula tells us how to generate this polynomial, and this is why we need the $x_0$ in the polynomial expansion.

$p(x) = f(x_0) + f'(x_0)(x-x_0) + \cdots+ \frac {f^{(n)}}{n!} (x-x_0)^n\cdots\\ p(x_0) = f(x_0) + f'(x_0)(x_0-x_0) + \cdots+ \frac {f^{(n)}}{n!} (x_0-x_0)^n\cdots = f(x_0)\\ p'(x) = f'(x_0) + f''(x-x_0) + \cdots+ \frac {f^{(n)}}{(n-1)!} (x-x_0)^{n-1}\cdots\\ p'(x_0) = f'(x_0)\\ p^{(n)}(x_0) = f^{(n)}(x_0)\\ $

Furthermore, this polynomial will be arbitrarily close to the original function everywhere inside the radius of convergence. That is for any finite polynomial the error is bounded, and the degree of the Taylor approximation increases the upper bound of the error term goes to $0.$

This implies that knowing all of the derivatives of a function as some point is sufficient information to know everything about that function.

Update:

The Taylor theorem says that there is a sequence of polynomials such that:

$p(x) = f(x_0) + f'(x_0)(x-x_0) + \cdots+ \frac {f^{(n)}}{n!} (x-x_0)^n\\ f(x) = p_n(x) + \epsilon(x)$

Where $\epsilon_n(x)$ is the error term.

If $f(x)$ is "smooth", i.e. all of its derivatives exists.

Over an interval

$|\epsilon_n(x)| < |\frac {f^{(n+1)}(\xi)}{(n+1)!} (x-x_0)^{n+1}|$

Where $\xi$ is chosen as the maximal value of $f^{(n+1)}(x)$ in that interval.

This puts an upper bound on the size of the error.

If the Taylor series converges, then $\lim_\limits{n\to\infty} |\frac {f^{(n+1)}(\xi)}{(n+1)!} (x-x_0)^{n+1}| =0$