3
$\begingroup$

Given a function $f: \mathbb {R} ^n \to \mathbb {R} $, I am curious about the uniqueness of a $k$th-order approximation at $c \in \mathbb {R}^n $, i.e. a function $\phi(x)$ such that

$$ \frac {f(c + h) - \phi(c + h)} {\| h \|^k } \overset{h\to 0} { \to } 0,$$

and I want to show that there is a unique such $\phi$ that is a polynomial of degree $\leq k$ in $(x_1 - c_1, \dotsc , x_n - c_n) = (h_1 , \dotsc, h_n)$ (supposing that one exists at all). Of course, this will be the Taylor expansion of order $k$ at $c$. We can prove the uniqueness of the first order as follows: if $p_1$, $p_1'$ are polynomials of degree $\leq 1$ in the components, then they are of the form $$p_1 = v + \langle w, h \rangle \\ p_1' = v' + \langle w', h \rangle,$$

so since $\frac {f- p_1} {\|h\|}$ and $ \frac {f - p_2} { \| h \| } \to 0$ as $h\to 0$, we have that

$$ \frac { p_2 - p_1 } { \| h \| } = \frac {v-v'} { \| h \| } +\frac { \langle w-w' , h \rangle } { \|h\| }, $$

and if we let $h \to 0$ parallel to $w-w'$, we see that this can go to zero iff $v = v', w = w'$.

If we try to do the same thing in second order, we'll end up with

$$ \frac {v - v'} {\| h \|^2 } + \frac { \langle w - w', h \rangle} {\| h\|^2} + \frac {h^T (H - H')h} {\|h \|^2} $$ goes to zero, and we'd like to conclude from this that $H = H'$ etc. I'm not exactly sure how we'd do this; we'd maybe like to use the matrix norm, but that gives an upper bound, and we need a lower bound (something to force to zero).

Anyway, my questions are:

  1. How can we show the uniqueness of such a $k$th order approximation in general?

  2. Given a $k$th order approximation which is also a polynomial of degree $\leq k$ in the $h_i$, how can we show that those terms which are of degree $\leq k-1$ form a $k-1$th-order approximation?

It would be a bonus if we can answer questions 1 and 2 without reference to the derivative, and then afterward we can of course show that the existence of such an approximation is equivalent to $k$-times differentiability at $c$ (sometimes it is even taken as the definition).

Edit: @40votes has pointed out that we cannot reason from the existence of such an approximation to the existence of a derivative of order higher than 1.

$\endgroup$
  • 1
    $\begingroup$ You can't get derivatives above 1st order from any kind of polynomial approximation. Consider $f(x)=x^{10}\sin(x^{-20})$, etc. $\endgroup$ – 40 votes Jul 24 '13 at 1:15
  • 2
    $\begingroup$ @TedShifrin The function $x^{10}\sin(x^{-20})$ admits a polynomial approximation of order $k=9$ at every point. It is not twice differentiable. $\endgroup$ – 40 votes Jul 24 '13 at 1:50
  • 1
    $\begingroup$ @40votes: You're right, of course. Eric, the correct reasoning is that if $f$ is $k$ times diff'able at $c$ and $P$ is a polynomial of degree $\le k$ that agrees with $f$ to order $k$ at $c$, then $P$ is the $k$th order Taylor polynomial. $\endgroup$ – Ted Shifrin Jul 24 '13 at 2:06
  • 1
    $\begingroup$ @EricAuld The zero polynomial approximates that function at $0$, and at other points you have the usual Taylor approximation. It's a variation of the standard $x^2\sin(1/x)$ function discussed in real analysis textbooks. $\endgroup$ – 40 votes Jul 24 '13 at 2:10
  • 1
    $\begingroup$ Eric, (2) should be a $1$-line argument. Think of $f(x)=P(x) + o(x^k)$. $\endgroup$ – Ted Shifrin Jul 24 '13 at 2:12
5
$\begingroup$

How can we show the uniqueness of such a $k$th order approximation in general?

If two polynomials $p,q$ of degree at most $k$ satisfy $$\lim_{h\to 0}\frac{p(c+h)-q(c+h)}{\|h\|^k}=0\tag1$$ for some $c$, then $p\equiv q$. Indeed, the difference $p-q$, being a polynomial of degree at most $k$, admits a finite Taylor expansion $$p(c+h)-q(c+h) = \sum_{|j|\le k} c_j h^j\tag2$$ where $j$ is a multiindex. Using (1) repeatedly with $k$ replaced by $0,1,2,\dots,k$ we find that $c_j=0$ for all $j$.

Given a $k$th order approximation which is also a polynomial of degree $\le k$, how can we show that those terms which are of degree $\le k−1$ form a $(k−1)$th-order approximation?

Given $p(c+h)=f(c+h)+o(\|h\|^k)$, write $p=q+p_k$ where $p_k$ contains the monomials in $h$ that are of degree precisely $k$, and $q$ has degree $<k$. Note that $p_{k}(c+h)=O(\|h\|^k)$ which is $o(\|h\|^{k-1})$. Hence $$p(c+h)-p_{k}(h)=f(c+h)+o(\|h\|^{k-1})$$


The existence of $k$th order approximation, for any $k\ge 1$, implies first-order differentiability. However, it does not imply the existence of second or higher order derivatives. A standard counterexample is $f(x)=x^{k+1}\sin x^{-k-1}$. The problem is that a bound on $f$ gives no control over the oscillation of $f'$. A more extreme example of this kind is $$f(x)=\exp (-x^{-2} ) \chi_{\mathbb Q}$$ which is discontinuous everywhere except $0$ but has polynomial approximation of all orders at $0$. (In both examples above, the function is extended to $x=0$ by $0$.)

In convex analysis and some areas of PDE one encounters pointwise second-order derivative, e.g., Second order differentiability of convex functions in Banach spaces by Borwein and Noll. These precisely correspond to the existence of 2nd order approximation described in the question. One can similarly define the pointwise $k$th derivative of $f$ as the corresponding derivative of its $k$th order polynomial approximation. I haven't seen this used for $k\ge 3$, although the concept is natural.

A related concept is jet.

$\endgroup$
  • $\begingroup$ Is it easy to prove that $p_k(x+h) = O(\|h\|^k)$? I thought $$\frac {x_1^{\alpha_1} \dotsb x_n^{\alpha_n} } {\|x\|^p}$$ had no limit at zero if $\sum \alpha_i = p$. Does it have an upper bound? $\endgroup$ – Eric Auld Jul 25 '13 at 23:40
  • 1
    $\begingroup$ @Eric It does not have a limit, but it does have an upper bound. First, restrict to the unit sphere and use compactness+continuity. Second, note that multiplying $x$ by a nonzero scalar does not change this quotient. $\endgroup$ – 40 votes Jul 25 '13 at 23:47
  • $\begingroup$ Below eq. (2) you say: "Using (1) repeatedly with k replaced by 0,1,2,…,k we find that cj=0 for all j." How do you conclude this? See my question about it: math.stackexchange.com/q/2148735. $\endgroup$ – Sjorszini Feb 19 '17 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.