# Is there an epsilon-delta definition of the second derivative?

Is there an epsilon-delta definition for the second derivative?

I know that there is such a definition for the first derivate $f'(x)$ which can be derived from the limit $f'(x) = \lim_{y\rightarrow x} \frac{f(y)-f(x)}{y-x}$ for a function $f:D\rightarrow \mathbb{R}$:

$$\forall \epsilon > 0\, \exists \delta > 0\, \forall y \in D\setminus \{x\}:|y-x|<\delta \Rightarrow \left|\frac{f(y)-f(x)}{y-x}-f'(x)\right|<\epsilon$$

So $f'(x)$ can be described as the number which fulfills the above statement. Is there a similar statement for the second derivative?

Update: This MSE thread shows that there are different definitions for the derivative (and thus for the second derivative). So I want to make my question more concrete:

My definition of derivation: Let be $f:D\rightarrow\mathbb{R}$ with $D\subseteq\mathbb{R}$ arbitrary. Let $D^*$ be the set off all points $x\in D$ for which there is at least one sequence $(x_n)$ in $D\setminus\{x\}$ with $\lim_{n\rightarrow\infty} x_n=x$. I define the limit $\lim_{y\rightarrow x\ ,y\in D\setminus\{x\}} {f(y)-f(x) \over y-x}$ as the first derivation for a given $x\in D^*$ (if the limit exists).

My definition of the second derivative: Let be $f:D\rightarrow\mathbb{R}$ with $D\subseteq\mathbb{R}$ arbitrary. We call $f''(x)$ the second derivative if there exists an open interval $x\in O\subseteq \mathbb{R}$ so that $f$ is differentiable on $O\cap D$ and $f''(x)$ is the first derivative of the function $f': (O\cap D)\rightarrow\mathbb{R}:x\mapsto f'(x)$ at the point $x$ (which also means that $x\in(O\cap D)^*$).

My question: Is there a statement $\forall \epsilon > 0: \exists \delta > 0: A(\epsilon, \delta, f, x, c)$ for $f:D\rightarrow \mathbb{R}$ ($D\subseteq \mathbb{R}$) and $c,x\in\mathbb{R}$ which is equivalent to the statement that $f$ is differentiable on a set $x\in O\cap D$ where $O$ is an open interval and that $c$ is the second derivative of $f$ at $x$?

I also will accept answers where you need more restrictions to the question. For example you might want to use the value of the first derivative $f'(x)$ (at the same point where you want to define the second derivative) in your statement or you want to restrict $f$ on functions with open domains or domains which are intervals. In this case I will accept your answer and open a new thread asking for a more general solution.

Please notice that there is a community wiki post where I want to collect all the progress we made so far.

-
Just get the first derivative, then use the definition on that one. =) – Pedro Tamaroff Jun 17 '12 at 14:45
@tampis It may be hard/unnatural considering the second derivative is defined as the derivative of the derivative. Perhaps one could nest one epsilon-delta definition into another but that is the kind of thing they only do to prisoners at Guantanamo Bay... – Ragib Zaman Jun 17 '12 at 14:54
@tampis: You must use at least the value of $f'(x)$. Otherwise how do you distinguish between functions with the same second derivative but not the same first derivative? – Zhen Lin Jun 17 '12 at 15:41
In order that the second derivative exist at $x$, it is necessary that the first derivative exist in an entire neighborhood of $x$. How can an epsilon-delta statement ensure that? – GEdgar Jun 19 '12 at 15:55
@RagibZaman And I thought that MSE was the ONE place I could go to get away from political BS. I guess I was wrong. – ItsNotObvious Jun 19 '12 at 15:58

I'm not sure, but I think there are two problems with the formula you used to approximate $f''(x_0)$:

• it uses the same discretization step for the approximation of the first and second derivative (it's like computing one directional derivative for a function of two variables: it might exist, but that does not imply that the differential exists).

• it's a centered finite difference formula, which therefore vanishes for a function that is odd around $x_0$ (or even gives infinity if $f$ is odd but $f(x_0)\neq0$, in which case $f$ is for sure discontinuous).

But I think the idea would work if the increments used in the approximation of the first and second derivative were different and the discretization formulas were not centered. Namely

$$f''(x)\simeq \frac{f'(x+h)-f'(x)}{h}$$

and then

$$f'(x+h)\simeq \frac{f(x+h+k)-f(x+h)}{k}$$ $$f'(x)\simeq \frac{f(x+k)-f(x)}{k}$$ which gives

$$f''(x)\simeq \frac{\dfrac{f(x+h+k)-f(x+h)}{k}-\dfrac{f(x+k)-f(x)}{k}}{h}=$$

$$f''(x)\simeq \frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}$$

Now, for the example reported in the link you gave, this formula does not give a finite result as $h,k$ go to $0$ independently.

To summarize, I would say that $f''$ exists and it's equal to $f''(x)$ if

$$\forall \varepsilon>0, \exists \delta>0 : \forall \underline{h}\in\mathbb{R}^2\cap \mathcal{B}(\underline{0},\delta)\setminus\underline{0}$$ $$\left|\frac{f(x+h_1+h_2)-f(x+h_1)-f(x+h_2)+f(x)}{h_1h_2}-f''(x)\right|<\varepsilon.$$

I'm not $100\%$ sure of this statement (in particular of the fact that the two increments have to independent), but it looks right to me. For sure you need not centered schemes though.

-
The centered difference formula is not a problem: for the first derivative to exist it can't be odd around our point $x_0$ unless it is zero. But you are right that $h$ and $k$ need to be able to vary independently. The counterexample OP cites will fail in this case. – Ross Millikan Jun 19 '12 at 18:08
I think that if you use centered schemes, even if $h$ and $k$ are independent, the counterexample that tampis posted still holds (if I did my calculations correctly). But I will double check it. – bartgol Jun 19 '12 at 21:26
I'm starting to believe that my definition of $f''$ in terms of $f$ is too restrictive. Indeed, if the condition above is satisfied $\forall\underline{h}\in\mathcal{B}(\underline{0},\delta)\setminus\underline{0}$‌​, taking the limit as $h_1$ goes to zero, we have the definition of $f''$ as the limit of the increment quotient of $f'$. So if that definition is satisfied, then we are fine. However, it might be too restrictive. Indeed, $f''$ is defined as the limit of the increment quotient of $f'$, that is when '$h_1$ has been already sent to zero'...in some sense. – bartgol Jun 19 '12 at 21:36
The increments $h_1$ and $h_2$ do have to be independent; otherwise you just reduce to $\lim_{h\to 0} (f(x+2h)-2f(x+h)+f(x)/h^2$, which is not sufficient for the second derivative to exist; consider e.g. $f(x) = x^3\sin(1/x)$ (with $f(0)=0$). – Mike Shulman May 13 '14 at 16:57
However, even with independent increments, this definition does not even imply that $f$ is continuous at $x$, as pointed out by Tom Goodwillie in his answer to mathoverflow.net/questions/165704/… --- if $f$ is a $\mathbb{Q}$-linear map $\mathbb{R}\to \mathbb{R}$, then the "second-order difference quotient" is identically zero. – Mike Shulman May 13 '14 at 16:59

If we are not allowed to talk about $f'(x)$ for $x\ne x_0$ it is not possible to talk about $f''(x_0)$ in the proper sense. One could, however, approach the idea of $f''(x_0)$ via the Taylor expansion of $f$ at $x_0$:

The function $f$, defined in a neighborhood of $x_0$ has second derivative $b$ at $x_0$ if there is an $a\in{\mathbb R}$ such that $$\lim_{h\to 0}{f(x_0+h)-f(x_0)- a h \over h^2}={b\over2}\ .$$ This $\lim$-condition can obviously be expanded into $\epsilon$-$\delta$-language.

Note, however, that the function $f(x):=x^3$ $(x\in{\mathbb Q})$ and $:=0$ $(x\notin{\mathbb Q})$ would have $f''(0)=0$ according to this definition.

-
Indeed, this defines what is known as "pointwise second derivative" , which exists more often than $f''(x)$. However, I question why you say "not possible". @bartgol suggested a plausible definition in terms of a limit in $\mathbb R^2$. So far we have no proof that it works, but no counterexample either. – user31373 Jun 19 '12 at 16:48
@Leonid Kovalev: Do you have any references for the concept of "pointwise second derivative$? A search in google just gave me links to articles which seemed to be really hard to understand for me. – tampis Jun 22 '12 at 9:08 @tampis Definition:$f$has a pointwise derivative of order$k$at point$a$if there exists a polynomial$p$of degree$k$such that$\lim_{x\to a}\frac{f(x)-p(x)}{(x-a)^k}=0$. If this holds, we define$f_{pt}^{(k)}(a)=p^{(k)}(a)$. When$k=1$, this is the same notion as$f'(a)$; but for$k>1$the existence of$f_{pt}^{(k)}(a)$does not imply the existence of$f^{(k)}(a)$... You will not find this notion in elementary textbooks, because it's not needed there. But there are situations, such as investigation of nonsmooth convex functions in$\mathbb R^n$, when it becomes useful (with$k=2$). – user31373 Jun 22 '12 at 14:50 In this community wiki post I want to collect all progress we made so far in answering this question. Please feel free to edit it and to extend it with your ideas (you can also start a new community wiki post if you have a new approaches) ## Notes ## Some approaches • I know the formula$f''(x) = \lim_{h\rightarrow 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}$but this limit does not provide the existence of the second derivative (see section “limit” of wikipedia article “second derivative”). So we cannot derive an epsilon-delta definition from the limit$\lim_{h\rightarrow 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}$. • There is the idea to apply the above epsilon-delta definition to the first derivative: $$\forall \epsilon > 0\, \exists \delta > 0\, \forall y \in D\setminus \{x\}:|y-x|<\delta \Rightarrow \left|\frac{f'(y)-f'(x)}{y-x}-f''(x)\right|<\epsilon$$ But then we already used the existence of the first derivative in our definition. To be a definition we shall conclude from it, that the function is differentiable in a neighborhood of$x$(This might be hard, I know ;-) ). • There is the approach by Christian Blatter to define the second derivative from the taylor series$f(x_0+h)=f(x_0)+f'(x)h+\tfrac 12 f''(x) h^2$, so that it should be the limit $$\lim_{h\to 0}2\cdot {f(x_0+h)-f(x_0)- a h \over h^2}=b$$ whereby$a$shall be the unique number for which the above limit exists. Unfortunately the function$f(x) := \begin{cases} x^3 & ;x\in\mathbb{Q} \\ 0&;x\notin\mathbb{Q} \end{cases}$is a function which is not differentiable in a neighborhood of$x$but the above limit exists for$a=0$. (Notice: the above formula gives a definition for the "pointwise second derivative", please see the comment of Leonid Kovalev) ## bartgol's approach bartgol has the following idea (see his answer): $$\forall \varepsilon>0, \exists \delta>0 : \forall (h,k)\in\mathbb{R}^2\cap \mathcal{B}(\underline{0},\delta)\setminus\underline{0}$$ $$\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}-f''(x)\right|<\varepsilon.$$ • Harald Hanche-Olsen mentioned the double mean value theorem in the comments. For me it seems to have a somehow similar form as bartgol's idea. • The counterexample$f(x) := \begin{cases} x^3 & ;x\in\mathbb{Q} \\ 0&;x\notin\mathbb{Q} \end{cases}$does not work for this approach ;-) Let$x=0$,$k+h\in\mathbb{Q}$with$k,h\notin\mathbb{Q}. Then we have \begin{align}\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}&=\frac{(h+k)^3}{hk}\\&=\frac{h^2}{k}+3h+3k+\frac{k^2}{h}\end{align} If we fixh<\delta$and let$k\rightarrow0$the amount of$\frac{h^2}{k}$will get arbitrary high, while$3h+3k+\frac{k^2}{h} \rightarrow 3h$so that$\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}-f''(x)\right|$cannot be smaller than$\varepsilon$. • I wanted to find a proof that$f$has to be differentiable in a neighborhood of$x$. Therefore there has to be an open interval$O$so that for all$x+h\in D\cap O$we have: $$\forall \epsilon > 0\, \exists \delta > 0\, \forall k,\tilde k\in\mathbb{R} : |k| < \delta \land |\tilde k |<\delta \land x+h+k,x+h+\tilde k\in D \Rightarrow \left|{f(x+h+k)-f(x+h) \over k} - {f(x+h+\tilde k)-f(x+h) \over \tilde k}\right|<\epsilon$$ With the representation$f(k+h+k)=f(x+h)+f(x+k)-f(x)+f''(x)hk+R(h,k)$($|R(h,k)|<|\epsilon h k|) I got \begin{align}&\left|{f(x+h+k)-f(x+h) \over k} - {f(x+h+\tilde k)-f(x+h) \over \tilde k}\right|\\=&\left|{f(x+k)-f(x)\over k }-{f(x+\tilde k)-f(x)\over \tilde k}+{R(h,k)\over k}-{R(h,\tilde k)\over \tilde k}\right|\end{align} If one can prove the differentiability off$at$x$the term${f(x+k)-f(x)\over k }-{f(x+\tilde k)-f(x)\over \tilde k}$can be made arbitrary small for$k,\tilde k\rightarrow 0$. Unfortunately we have$\left|{R(h,k)\over k}\right|<|\epsilon h|$so that we have no control over this term if$k$goes to zero. • My interpretation of bartgol's idea for arbitrary functions is (let be$f:D\rightarrow \mathbb{R}$): Forall$\epsilon > 0$, there is a$\delta > 0$, so that for all$x+h+k\in D$with$|h|+|k|<\delta$we have $$f(x+h+k)=f(x+h)+f(x+k) -f(x)+f''(x)hk+R(h,k)$$ so that$|R(h,k)|<|\varepsilon h k|$. • There is a counterexample for the above interpretation of bartgol's idea (Please notice, that bartgol's idea may still work for functions with open domains): Let$A:=\{q\cdot\pi : q\in\mathbb{Q}\}$and$B:=\{q\cdot\sqrt{2} : q\in\mathbb{Q}\}$.$A$and$B$are abelian groups with$+$as group operation and$A\cap B=\{0\}$. If we just consider$k,h\in (A \cup B)\setminus\{0\}$we have$k+h\in A \Rightarrow k\in A \land h \in A$and$k+h\in B \Rightarrow k\in B \land h \in B$. Now we define$f:A\cup B\rightarrow \mathbb{R}: x \mapsto \begin{cases} x^3 & ;x\in A\setminus\{0\} \\ 0 & ; x\in B\end{cases}$. For$x=0we have either: \begin{align}\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}\right|&=\left|\frac{(h+k)^3-h^3-k^3}{hk}\right|\\&=\left|3h+3k\right|\\&\le 3 (|h|+|k|)\end{align} or $$\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}\right|=\left|\frac{0}{hk}\right|=0$$ This proves that the above statement is true forf$with$f''(0)=0$. Because$A$and$B$are dense in$\mathbb{R}$the concept of derivation is well defined for$f$. But because there are noncontinuous jumps in every neighborhood of a point$x\in A \cup B\setminus \{0\}$the function$f$is not differentiable in a neighborhood of$x=0$. • Actually, this definition does not work for functions defined on open domains either. Consider a$\mathbb{Q}$-linear function$f:\mathbb{R}\to \mathbb{R}$. Then$f(x+h+k)-f(x+h)-f(x+k)+f(x)$is identically zero, so this definition would give$f''(0)=0$; but such a function need not even be continuous. This was pointed out by Tom Goodwillie here. He did show, however, that this definition does work if we assume that$f$is differentiable in a neighborhood. - Here is an enhancement of bartgol's idea which also implies that the function is differentiable at$x$. There is a number$a$such that$\forall \epsilon>0 \;\exists \delta>0$such that if$h^2 h^k + v^2 <\delta$, then $$\Big| f(x+h+k+v) - f(x+h) - f(x+k) + f(x) - c h k - a v \Big| < \epsilon.$$ Letting$v=0$we recover bartgol's definition. But letting$h=k=0$, we recover the statement that$f$is differentiable at$x$with derivative$a$. Therefore, if$f$satisfies this definition, then it is differentiable at$x$; and furthermore if it is differentiable in some neighborhood of$x$, then it is twice differentiable at$x$and$f''(x)=c\$.

-
Thanks for your comment! – tampis Aug 1 '14 at 20:41

I think I can prove it must exist; however this proof says nothing about the ability to write it down (as you know, there are functions with no elemental form).

We know from physics that there is a such thing as a position, and velocity is the change of position, and acceleration is the change of change of position. Let us say that we are pre-Newton, and we failed initially to find the closed form to compute the trajectory of a falling object in a vacuum. But we have Galileo's famous experiment on the Leaning Tower of Pisa. It is hard, but not impossible to have determined what we know to be true, that g is a fixed acceleration.

We estimate z(t) ~= z(t - ϵ) + z'(t - ϵ) ; z'(t) ~= z'(t - ϵ) + g * ϵ

By tightening up ϵ this approximation becomes more accurate.

It should quickly become obvious that ∀ϵ < k ∃δ |z(t)(real) - z(t)(estimated)| < δ where k is of the same order as g. Generalizing this will lead to the second order epsilon-delta definition.

Gentlemen, I have come to the conclusion this is vitally important. This works because the definitions of calculus mean something. They are not arbitrary but rational and are effective tools.

-
As I cannot do the generalization in the last step I cannot say if we can write down the final form. – Joshua Jul 24 '15 at 2:07