Is there an epsilon-delta definition for the second derivative?

I know that there is such a definition for the first derivate $f'(x)$ which can be derived from the limit $f'(x) = \lim_{y\rightarrow x} \frac{f(y)-f(x)}{y-x}$ for a function $f:D\rightarrow \mathbb{R}$:

$$\forall \epsilon > 0\, \exists \delta > 0\, \forall y \in D\setminus \{x\}:|y-x|<\delta \Rightarrow \left|\frac{f(y)-f(x)}{y-x}-f'(x)\right|<\epsilon$$

So $f'(x)$ can be described as the number which fulfills the above statement. Is there a similar statement for the second derivative?

Update: This MSE thread shows that there are different definitions for the derivative (and thus for the second derivative). So I want to make my question more concrete:

My definition of derivation: Let be $f:D\rightarrow\mathbb{R}$ with $D\subseteq\mathbb{R}$ arbitrary. Let $D^*$ be the set off all points $x\in D$ for which there is at least one sequence $(x_n)$ in $D\setminus\{x\}$ with $\lim_{n\rightarrow\infty} x_n=x$. I define the limit $\lim_{y\rightarrow x\ ,y\in D\setminus\{x\}} {f(y)-f(x) \over y-x}$ as the first derivation for a given $x\in D^*$ (if the limit exists).

My definition of the second derivative: Let be $f:D\rightarrow\mathbb{R}$ with $D\subseteq\mathbb{R}$ arbitrary. We call $f''(x)$ the second derivative if there exists an open interval $x\in O\subseteq \mathbb{R}$ so that $f$ is differentiable on $O\cap D$ and $f''(x)$ is the first derivative of the function $f': (O\cap D)\rightarrow\mathbb{R}:x\mapsto f'(x)$ at the point $x$ (which also means that $x\in(O\cap D)^*$).

My question: Is there a statement $\forall \epsilon > 0: \exists \delta > 0: A(\epsilon, \delta, f, x, c)$ for $f:D\rightarrow \mathbb{R}$ ($D\subseteq \mathbb{R}$) and $c,x\in\mathbb{R}$ which is equivalent to the statement that $f$ is differentiable on a set $x\in O\cap D$ where $O$ is an open interval and that $c$ is the second derivative of $f$ at $x$?

I also will accept answers where you need more restrictions to the question. For example you might want to use the value of the first derivative $f'(x)$ (at the same point where you want to define the second derivative) in your statement or you want to restrict $f$ on functions with open domains or domains which are intervals. In this case I will accept your answer and open a new thread asking for a more general solution.

Please notice that there is a community wiki post where I want to collect all the progress we made so far.

  • 7
    $\begingroup$ Just get the first derivative, then use the definition on that one. =) $\endgroup$ – Pedro Tamaroff Jun 17 '12 at 14:45
  • 5
    $\begingroup$ @tampis It may be hard/unnatural considering the second derivative is defined as the derivative of the derivative. Perhaps one could nest one epsilon-delta definition into another but that is the kind of thing they only do to prisoners at Guantanamo Bay... $\endgroup$ – Ragib Zaman Jun 17 '12 at 14:54
  • 3
    $\begingroup$ @tampis: You must use at least the value of $f'(x)$. Otherwise how do you distinguish between functions with the same second derivative but not the same first derivative? $\endgroup$ – Zhen Lin Jun 17 '12 at 15:41
  • 3
    $\begingroup$ In order that the second derivative exist at $x$, it is necessary that the first derivative exist in an entire neighborhood of $x$. How can an epsilon-delta statement ensure that? $\endgroup$ – GEdgar Jun 19 '12 at 15:55
  • 4
    $\begingroup$ @RagibZaman And I thought that MSE was the ONE place I could go to get away from political BS. I guess I was wrong. $\endgroup$ – ItsNotObvious Jun 19 '12 at 15:58

I'm not sure, but I think there are two problems with the formula you used to approximate $f''(x_0)$:

  • it uses the same discretization step for the approximation of the first and second derivative (it's like computing one directional derivative for a function of two variables: it might exist, but that does not imply that the differential exists).

  • it's a centered finite difference formula, which therefore vanishes for a function that is odd around $x_0$ (or even gives infinity if $f$ is odd but $f(x_0)\neq0$, in which case $f$ is for sure discontinuous).

But I think the idea would work if the increments used in the approximation of the first and second derivative were different and the discretization formulas were not centered. Namely

$$f''(x)\simeq \frac{f'(x+h)-f'(x)}{h}$$

and then

$$f'(x+h)\simeq \frac{f(x+h+k)-f(x+h)}{k}$$ $$f'(x)\simeq \frac{f(x+k)-f(x)}{k}$$ which gives

$$f''(x)\simeq \frac{\dfrac{f(x+h+k)-f(x+h)}{k}-\dfrac{f(x+k)-f(x)}{k}}{h}=$$

$$f''(x)\simeq \frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}$$

Now, for the example reported in the link you gave, this formula does not give a finite result as $h,k$ go to $0$ independently.

To summarize, I would say that $f''$ exists and it's equal to $f''(x)$ if

$$\forall \varepsilon>0, \exists \delta>0 : \forall \underline{h}\in\mathbb{R}^2\cap \mathcal{B}(\underline{0},\delta)\setminus\underline{0}$$ $$\left|\frac{f(x+h_1+h_2)-f(x+h_1)-f(x+h_2)+f(x)}{h_1h_2}-f''(x)\right|<\varepsilon.$$

I'm not $100\%$ sure of this statement (in particular of the fact that the two increments have to independent), but it looks right to me. For sure you need not centered schemes though.

  • 1
    $\begingroup$ The centered difference formula is not a problem: for the first derivative to exist it can't be odd around our point $x_0$ unless it is zero. But you are right that $h$ and $k$ need to be able to vary independently. The counterexample OP cites will fail in this case. $\endgroup$ – Ross Millikan Jun 19 '12 at 18:08
  • $\begingroup$ I think that if you use centered schemes, even if $h$ and $k$ are independent, the counterexample that tampis posted still holds (if I did my calculations correctly). But I will double check it. $\endgroup$ – bartgol Jun 19 '12 at 21:26
  • $\begingroup$ I'm starting to believe that my definition of $f''$ in terms of $f$ is too restrictive. Indeed, if the condition above is satisfied $\forall\underline{h}\in\mathcal{B}(\underline{0},\delta)\setminus\underline{0}$, taking the limit as $h_1$ goes to zero, we have the definition of $f''$ as the limit of the increment quotient of $f'$. So if that definition is satisfied, then we are fine. However, it might be too restrictive. Indeed, $f''$ is defined as the limit of the increment quotient of $f'$, that is when '$h_1$ has been already sent to zero'...in some sense. $\endgroup$ – bartgol Jun 19 '12 at 21:36
  • 1
    $\begingroup$ The increments $h_1$ and $h_2$ do have to be independent; otherwise you just reduce to $\lim_{h\to 0} (f(x+2h)-2f(x+h)+f(x)/h^2$, which is not sufficient for the second derivative to exist; consider e.g. $f(x) = x^3\sin(1/x)$ (with $f(0)=0$). $\endgroup$ – Mike Shulman May 13 '14 at 16:57
  • 1
    $\begingroup$ However, even with independent increments, this definition does not even imply that $f$ is continuous at $x$, as pointed out by Tom Goodwillie in his answer to mathoverflow.net/questions/165704/… --- if $f$ is a $\mathbb{Q}$-linear map $\mathbb{R}\to \mathbb{R}$, then the "second-order difference quotient" is identically zero. $\endgroup$ – Mike Shulman May 13 '14 at 16:59

If we are not allowed to talk about $f'(x)$ for $x\ne x_0$ it is not possible to talk about $f''(x_0)$ in the proper sense. One could, however, approach the idea of $f''(x_0)$ via the Taylor expansion of $f$ at $x_0$:

The function $f$, defined in a neighborhood of $x_0$ has second derivative $b$ at $x_0$ if there is an $a\in{\mathbb R}$ such that $$\lim_{h\to 0}{f(x_0+h)-f(x_0)- a h \over h^2}={b\over2}\ .$$ This $\lim$-condition can obviously be expanded into $\epsilon$-$\delta$-language.

Note, however, that the function $f(x):=x^3$ $(x\in{\mathbb Q})$ and $:=0$ $(x\notin{\mathbb Q})$ would have $f''(0)=0$ according to this definition.

  • $\begingroup$ Indeed, this defines what is known as "pointwise second derivative" , which exists more often than $f''(x)$. However, I question why you say "not possible". @bartgol suggested a plausible definition in terms of a limit in $\mathbb R^2$. So far we have no proof that it works, but no counterexample either. $\endgroup$ – user31373 Jun 19 '12 at 16:48
  • 2
    $\begingroup$ @Leonid Kovalev: Do you have any references for the concept of "pointwise second derivative$? A search in google just gave me links to articles which seemed to be really hard to understand for me. $\endgroup$ – Stephan Kulla Jun 22 '12 at 9:08
  • $\begingroup$ @tampis Definition: $f$ has a pointwise derivative of order $k$ at point $a$ if there exists a polynomial $p$ of degree $k$ such that $\lim_{x\to a}\frac{f(x)-p(x)}{(x-a)^k}=0$. If this holds, we define $f_{pt}^{(k)}(a)=p^{(k)}(a)$. When $k=1$, this is the same notion as $f'(a)$; but for $k>1$ the existence of $f_{pt}^{(k)}(a)$ does not imply the existence of $f^{(k)}(a)$... You will not find this notion in elementary textbooks, because it's not needed there. But there are situations, such as investigation of nonsmooth convex functions in $\mathbb R^n$, when it becomes useful (with $k=2$). $\endgroup$ – user31373 Jun 22 '12 at 14:50
  • $\begingroup$ @user31373 : It seems to me that this definition should explicitly require $f$ to be continuous at $a$; the definition as you state it says nothing at all about $f(a)$, but we cannot recreate the usual notion for $k = 1$ without this. (Then pointwise differentiability is weaker than differentiability for $k > 1$, equivalent for $k = 1$, and stronger for $k = 0$, since even 0-times pointwise differentiability explicitly requires continuity.) $\endgroup$ – Toby Bartels Oct 14 '17 at 4:03
  • $\begingroup$ While I'm commenting, I like to think that the $pt$ in $f^{(k)}_{pt}$ stands for Peano–Taylor; the fact that the pointwise derivative equals the derivative when the latter exists is essentially Taylor's theorem with the Peano form of the remainder. (The polynomial $p$ in user31373's definition must be the $k$th-order Taylor polynomial.) $\endgroup$ – Toby Bartels Oct 14 '17 at 4:06

In this community wiki post I want to collect all progress we made so far in answering this question. Please feel free to edit it and to extend it with your ideas (you can also start a new community wiki post if you have a new approaches)


Some approaches

  • I know the formula $f''(x) = \lim_{h\rightarrow 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}$ but this limit does not provide the existence of the second derivative (see section “limit” of wikipedia article “second derivative”). So we cannot derive an epsilon-delta definition from the limit $\lim_{h\rightarrow 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}$.

  • There is the idea to apply the above epsilon-delta definition to the first derivative:

    $$\forall \epsilon > 0\, \exists \delta > 0\, \forall y \in D\setminus \{x\}:|y-x|<\delta \Rightarrow \left|\frac{f'(y)-f'(x)}{y-x}-f''(x)\right|<\epsilon$$

    But then we already used the existence of the first derivative in our definition. To be a definition we shall conclude from it, that the function is differentiable in a neighborhood of $x$ (This might be hard, I know ;-) ).

  • There is the approach by Christian Blatter to define the second derivative from the taylor series $f(x_0+h)=f(x_0)+f'(x)h+\tfrac 12 f''(x) h^2$, so that it should be the limit

    $$\lim_{h\to 0}2\cdot {f(x_0+h)-f(x_0)- a h \over h^2}=b$$

    whereby $a$ shall be the unique number for which the above limit exists. Unfortunately the function $f(x) := \begin{cases} x^3 & ;x\in\mathbb{Q} \\ 0&;x\notin\mathbb{Q} \end{cases}$ is a function which is not differentiable in a neighborhood of $x$ but the above limit exists for $a=0$. (Notice: the above formula gives a definition for the "pointwise second derivative", please see the comment of Leonid Kovalev)

bartgol's approach

bartgol has the following idea (see his answer):

$$\forall \varepsilon>0, \exists \delta>0 : \forall (h,k)\in\mathbb{R}^2\cap \mathcal{B}(\underline{0},\delta)\setminus\underline{0}$$ $$\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}-f''(x)\right|<\varepsilon.$$

  • Harald Hanche-Olsen mentioned the double mean value theorem in the comments. For me it seems to have a somehow similar form as bartgol's idea.

  • The counterexample $f(x) := \begin{cases} x^3 & ;x\in\mathbb{Q} \\ 0&;x\notin\mathbb{Q} \end{cases}$ does not work for this approach ;-) Let $x=0$, $k+h\in\mathbb{Q}$ with $k,h\notin\mathbb{Q}$. Then we have


    If we fix $h<\delta$ and let $k\rightarrow0$ the amount of $\frac{h^2}{k}$ will get arbitrary high, while $3h+3k+\frac{k^2}{h} \rightarrow 3h$ so that $\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}-f''(x)\right|$ cannot be smaller than $\varepsilon$.

  • I wanted to find a proof that $f$ has to be differentiable in a neighborhood of $x$. Therefore there has to be an open interval $O$ so that for all $x+h\in D\cap O$ we have:

    $$\forall \epsilon > 0\, \exists \delta > 0\, \forall k,\tilde k\in\mathbb{R} : |k| < \delta \land |\tilde k |<\delta \land x+h+k,x+h+\tilde k\in D \Rightarrow \left|{f(x+h+k)-f(x+h) \over k} - {f(x+h+\tilde k)-f(x+h) \over \tilde k}\right|<\epsilon$$

    With the representation $f(k+h+k)=f(x+h)+f(x+k)-f(x)+f''(x)hk+R(h,k)$ ($|R(h,k)|<|\epsilon h k|$) I got

    $$\begin{align}&\left|{f(x+h+k)-f(x+h) \over k} - {f(x+h+\tilde k)-f(x+h) \over \tilde k}\right|\\=&\left|{f(x+k)-f(x)\over k }-{f(x+\tilde k)-f(x)\over \tilde k}+{R(h,k)\over k}-{R(h,\tilde k)\over \tilde k}\right|\end{align}$$

    If one can prove the differentiability of $f$ at $x$ the term ${f(x+k)-f(x)\over k }-{f(x+\tilde k)-f(x)\over \tilde k}$ can be made arbitrary small for $k,\tilde k\rightarrow 0$. Unfortunately we have $\left|{R(h,k)\over k}\right|<|\epsilon h|$so that we have no control over this term if $k$ goes to zero.

  • My interpretation of bartgol's idea for arbitrary functions is (let be $f:D\rightarrow \mathbb{R}$): Forall $\epsilon > 0$, there is a $\delta > 0$, so that for all $x+h+k\in D$ with $|h|+|k|<\delta$ we have

    $$f(x+h+k)=f(x+h)+f(x+k) -f(x)+f''(x)hk+R(h,k)$$

    so that $|R(h,k)|<|\varepsilon h k|$.

    • There is a counterexample for the above interpretation of bartgol's idea (Please notice, that bartgol's idea may still work for functions with open domains): Let $A:=\{q\cdot\pi : q\in\mathbb{Q}\}$ and $B:=\{q\cdot\sqrt{2} : q\in\mathbb{Q}\}$. $A$ and $B$ are abelian groups with $+$ as group operation and $A\cap B=\{0\}$. If we just consider $k,h\in (A \cup B)\setminus\{0\}$ we have $k+h\in A \Rightarrow k\in A \land h \in A$ and $k+h\in B \Rightarrow k\in B \land h \in B$. Now we define $f:A\cup B\rightarrow \mathbb{R}: x \mapsto \begin{cases} x^3 & ;x\in A\setminus\{0\} \\ 0 & ; x\in B\end{cases}$. For $x=0$ we have either:

      $$\begin{align}\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}\right|&=\left|\frac{(h+k)^3-h^3-k^3}{hk}\right|\\&=\left|3h+3k\right|\\&\le 3 (|h|+|k|)\end{align}$$



      This proves that the above statement is true for $f$ with $f''(0)=0$. Because $A$ and $B$ are dense in $\mathbb{R}$ the concept of derivation is well defined for $f$. But because there are noncontinuous jumps in every neighborhood of a point $x\in A \cup B\setminus \{0\}$ the function $f$ is not differentiable in a neighborhood of $x=0$.

  • Actually, this definition does not work for functions defined on open domains either. Consider a $\mathbb{Q}$-linear function $f:\mathbb{R}\to \mathbb{R}$. Then $f(x+h+k)-f(x+h)-f(x+k)+f(x)$ is identically zero, so this definition would give $f''(0)=0$; but such a function need not even be continuous. This was pointed out by Tom Goodwillie here. He did show, however, that this definition does work if we assume that $f$ is differentiable in a neighborhood.


Here is an enhancement of bartgol's idea which also implies that the function is differentiable at $x$. There is a number $a$ such that $\forall \epsilon>0 \;\exists \delta>0$ such that if $h^2 h^k + v^2 <\delta$, then

$$ \Big| f(x+h+k+v) - f(x+h) - f(x+k) + f(x) - c h k - a v \Big| < \epsilon. $$

Letting $v=0$ we recover bartgol's definition. But letting $h=k=0$, we recover the statement that $f$ is differentiable at $x$ with derivative $a$. Therefore, if $f$ satisfies this definition, then it is differentiable at $x$; and furthermore if it is differentiable in some neighborhood of $x$, then it is twice differentiable at $x$ and $f''(x)=c$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.