I am reading one-variable calculus book where it explains Taylor series and little confused with the following terms:

(1) Expand $f(x)$ at $x_0$
(2) Evaluate $f(x)$ at x
(3) Best Affine, linear, quadratic, cubic, and other approximations

My understanding is roughly...:
(1) Express an arbitrary function $f(x)$ in terms of polynomials with various terms of coefficients being the derivative values all evaluated at the point of expansion $x_0$.
(2) Given you have Taylor series from (1), the RHS is basically $n^{th}$degree polynomial with argument $x$. So just plug in a number in $x$ if you want to "evaluate" $f$.
(3) There is a class of functions you can use Taylor's formula to get the $n^{th}$ degree polynomial which is exactly $f(x)$. But this may not be the case, and in that case, we have Taylor's formula with remainder indicating the polynomial form is only approximation. The best affine approximation is of the form $f(x_o)+f'(x_o)(x-x_o)$ where as the linear approximation is $f'(x_o)$. Notch up one to $n=2$, you have power term so it is quadratic, and so forth.

My questions:

(1) I understand the goal is turning an arbitrarily function to a polynomial expression, but I don't quite get how this "expand" and "evaluate" machinery works as big picture.
(2) I remember in high school, teacher would say "we can ignore the remainder term in Taylor's Theorem, because it is so small". To what class of functions does this apply, and when is an exception to this rule?
(3) In introductory calculus book, it just mentions polynomials are easier to work with and have nice properties, but in introductory analysis course, I know it goes beyond this with power series, uniform convergence, convolutions, Borel Theorem, and the Weierstrass approximation. Can someone provide a big picture from the beginning to Weierstrass? I simply want to look at the whole theme of "approximation" and in the context of optimization. The main motivation from the univariate calculus book in covering Taylor is to develop relative extrema test. So I am reading the section in the context of optimization.

Thank you.

  • $\begingroup$ #1 means "write down the Taylor series (/ Taylor polynomial of relevant degree) for $f$ at $x_0$". #2 is nonsensical. $f(x)$ is already evaluated at $x$ so I don't know what your book is trying to say. #3 are just other terms for the degree $1$ Taylor polynomial (affine/ linear), degree $2$ Taylor polynomial (quadratic), and degree $3$ Taylor polynomial (cubic). Taylor polynomials are just approximations for a function -- unless that function happens to be a polynomial itself. $\endgroup$ – user137731 Jun 24 '16 at 21:29
  • $\begingroup$ @Bye_World, thanks for the comment. Here's my understanding about expanding and evaluating. Please correct me if I am wrong. You are thrown an arbitrary function at asked to "express" this in terms of polynomials at a point $x_0$. To do this, you assume continuous differentiability of the arbitrary function so you obtain all derivatives you need. So the more (e.g. higher terms) derivatives you have in your Taylor expansion, the closer your polynomial will behave locally at $x_0$. $\endgroup$ – Frank Swanton Jun 27 '16 at 13:43
  • $\begingroup$ @Bye_World, Given this, you basically have a polynomial function approximating the original arbitrary function, but this polynomial function has the same argument $x$. So in theory, you can punch in any $x$ in domain, suppose far from $x_0$. Then, there are several issues. For example, Taylor polynomials may or may not converge at the point of expansion. Even if it does, it only captures the local behavior accurately, so it you are "evaluating" a point far from that point of expansion, there is no guarantee your "approximation" is accurate. $\endgroup$ – Frank Swanton Jun 27 '16 at 13:46
  • $\begingroup$ @Bye_World, if the Taylor series is not convergent to the arbitrary function at the point of expansion as $n\rightarrow\infty$, then this is an issue, right? Not sure how you proceed from here. $\endgroup$ – Frank Swanton Jun 27 '16 at 13:47

I'll try to address some of the misunderstandings I think you have based on the question and the comments. Someone else can tackle the problem given in $(3)$ of explaining everything about Taylor polynomials and Taylor series from the ground up if they want (though I doubt anyone will).

First off, it sounds like you're conflating Taylor polynomials with Taylor series. A Taylor polynomial is just what it sounds like -- a polynomial. How can a polynomial "not converge"? Polynomials are defined everywhere.

Taylor's theorem says that a function which is $k$-times differentiable at a point $x$ can be approximated in some neighborhood of the point $x$ by the $k$th degree Taylor polynomial of the function centered at $x$. And depending on how you prove the theorem, you have different ways of bounding the error in your approximation. You're right the Taylor polynomials are only good approximations in some neighborhood of the point $x$ -- it depends on the function and your tolerance in the error as to how big that neighborhood is.

One can approximate a function by a Taylor series if it is infinitely differentiable at the point $x$. To do this one considers the sequence of degree $k$ Taylor polynomials as $k\to \infty$. But even if that sequence converges that doesn't mean that the function necessarily equals its Taylor series even in some small neighborhood of the point $x$. Sometimes the remainder terms don't tend to $0$ even in the limit. A function which does equal its Taylor series in some neighborhood of every point is called an analytic function.

The terms "best ____ approximation" mean the function of type ____ whose remainder term vanishes fastest near the point $x$. For example, the best affine approximation of a function $f$ near $x_0$ is the function $L(x) = f(x_0) + f'(x_0)(x-x_0)$. The reason this is the best is that of all of the affine functions $T(x) = A+Bx$, the above is the only one such that $\lim_{x\to x_0} \frac{f(x)-T(x)}{x-x_0} = \lim_{x\to x_0} \frac{\text{remainder}}{x-x_0}=0$. It turns out that the Taylor polynomials are always the best polynomial approximations of a function in some neighborhood of $x$.

One other note: continuous differentiability does not mean infinitely differentiable. If a function is continously differentiable at a point that means that it is differentiable and its derivative is continuous. But if a function is twice differentiable -- without even having a continuous second derivative -- then it is automatically continuously differentiable because differentiability implies continuity. So if it's twice differentiable then that means that its derivative is differentiable which means that the derivative is continuous. So don't mix up those ideas. Any $k$-times differentiable function is at least $(k-1)$-times continuously differentiable.

  • $\begingroup$ This is excellent. Thank you for the answer. It brings clarity. $\endgroup$ – Frank Swanton Jun 27 '16 at 14:51
  • $\begingroup$ Bye_World, when you say, for example, $f:\mathbb{R^N}\rightarrow\mathbb{R}$, $f$ is "twice differentiable", then you can obtain all second-degree partials on $f$ and since differentiability implies continuity, this case only guarantees the continuity of first partials? Then, if we say $f$ is "twice continuously differentiable" then it is everything above plus the second partials are continuous? So if we represented as class of functions, it would be $C^2\subset D^2\subset C^1\subset D^1\subset C$? $\endgroup$ – Frank Swanton Jun 28 '16 at 14:06
  • $\begingroup$ Also, how should I make sense of "Taylor expansion" in light of Taylor polynomial and Taylor series? Should I understand this as the sum of $n^{th}$ degree Taylor polynomial and the remainder term? $\endgroup$ – Frank Swanton Jun 28 '16 at 14:11
  • 1
    $\begingroup$ Saying a function is continuously differentiable is stronger than saying that its partials are all continuous. But otherwise you're correct. "Taylor expansion" is an ambiguous term. Usually it either means Taylor series or the lowest degree Taylor polynomial that we can use for the particular problem at hand (with or without the remainder term). $\endgroup$ – user137731 Jun 28 '16 at 14:20
  • 1
    $\begingroup$ If a function is continuously differentiable then its total derivative exists and is continuous. This implies that all of its partials are continuous, but even existence of all of the partials does not imply existence of the total derivative. $\endgroup$ – user137731 Jun 28 '16 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.