Is there an epsilon-delta definition for the second derivative?

I know that there is such a definition for the first derivate $f'(x)$ which can be derived from the limit $f'(x) = \lim_{y\rightarrow x} \frac{f(y)-f(x)}{y-x}$ for a function $f:D\rightarrow \mathbb{R}$:

$$\forall \epsilon > 0\, \exists \delta > 0\, \forall y \in D\setminus \{x\}:|y-x|<\delta \Rightarrow \left|\frac{f(y)-f(x)}{y-x}-f'(x)\right|<\epsilon$$

So $f'(x)$ can be described as the number which fulfills the above statement. Is there a similar statement for the second derivative?

Update: This MSE thread shows that there are different definitions for the derivative (and thus for the second derivative). So I want to make my question more concrete:

My definition of derivation: Let be $f:D\rightarrow\mathbb{R}$ with $D\subseteq\mathbb{R}$ arbitrary. Let $D^*$ be the set off all points $x\in D$ for which there is at least one sequence $(x_n)$ in $D\setminus\{x\}$ with $\lim_{n\rightarrow\infty} x_n=x$. I define the limit $\lim_{y\rightarrow x\ ,y\in D\setminus\{x\}} {f(y)-f(x) \over y-x}$ as the first derivation for a given $x\in D^*$ (if the limit exists).

My definition of the second derivative: Let be $f:D\rightarrow\mathbb{R}$ with $D\subseteq\mathbb{R}$ arbitrary. We call $f''(x)$ the second derivative if there exists an open interval $x\in O\subseteq \mathbb{R}$ so that $f$ is differentiable on $O\cap D$ and $f''(x)$ is the first derivative of the function $f': (O\cap D)\rightarrow\mathbb{R}:x\mapsto f'(x)$ at the point $x$ (which also means that $x\in(O\cap D)^*$).

My question: Is there a statement $\forall \epsilon > 0: \exists \delta > 0: A(\epsilon, \delta, f, x, c)$ for $f:D\rightarrow \mathbb{R}$ ($D\subseteq \mathbb{R}$) and $c,x\in\mathbb{R}$ which is equivalent to the statement that $f$ is differentiable on a set $x\in O\cap D$ where $O$ is an open interval and that $c$ is the second derivative of $f$ at $x$?

I also will accept answers where you need more restrictions to the question. For example you might want to use the value of the first derivative $f'(x)$ (at the same point where you want to define the second derivative) in your statement or you want to restrict $f$ on functions with open domains or domains which are intervals. In this case I will accept your answer and open a new thread asking for a more general solution.

Please notice that there is a community wiki post where I want to collect all the progress we made so far.

  • 7
    $\begingroup$ Just get the first derivative, then use the definition on that one. =) $\endgroup$
    – Pedro
    Jun 17, 2012 at 14:45
  • 5
    $\begingroup$ @tampis It may be hard/unnatural considering the second derivative is defined as the derivative of the derivative. Perhaps one could nest one epsilon-delta definition into another but that is the kind of thing they only do to prisoners at Guantanamo Bay... $\endgroup$ Jun 17, 2012 at 14:54
  • 3
    $\begingroup$ @tampis: You must use at least the value of $f'(x)$. Otherwise how do you distinguish between functions with the same second derivative but not the same first derivative? $\endgroup$
    – Zhen Lin
    Jun 17, 2012 at 15:41
  • 3
    $\begingroup$ In order that the second derivative exist at $x$, it is necessary that the first derivative exist in an entire neighborhood of $x$. How can an epsilon-delta statement ensure that? $\endgroup$
    – GEdgar
    Jun 19, 2012 at 15:55
  • 4
    $\begingroup$ @RagibZaman And I thought that MSE was the ONE place I could go to get away from political BS. I guess I was wrong. $\endgroup$ Jun 19, 2012 at 15:58

4 Answers 4


I'm not sure, but I think there are two problems with the formula you used to approximate $f''(x_0)$:

  • it uses the same discretization step for the approximation of the first and second derivative (it's like computing one directional derivative for a function of two variables: it might exist, but that does not imply that the differential exists).

  • it's a centered finite difference formula, which therefore vanishes for a function that is odd around $x_0$ (or even gives infinity if $f$ is odd but $f(x_0)\neq0$, in which case $f$ is for sure discontinuous).

But I think the idea would work if the increments used in the approximation of the first and second derivative were different and the discretization formulas were not centered. Namely

$$f''(x)\simeq \frac{f'(x+h)-f'(x)}{h}$$

and then

$$f'(x+h)\simeq \frac{f(x+h+k)-f(x+h)}{k}$$ $$f'(x)\simeq \frac{f(x+k)-f(x)}{k}$$ which gives

$$f''(x)\simeq \frac{\dfrac{f(x+h+k)-f(x+h)}{k}-\dfrac{f(x+k)-f(x)}{k}}{h}=$$

$$f''(x)\simeq \frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}$$

Now, for the example reported in the link you gave, this formula does not give a finite result as $h,k$ go to $0$ independently.

To summarize, I would say that $f''$ exists and it's equal to $f''(x)$ if

$$\forall \varepsilon>0, \exists \delta>0 : \forall \underline{h}\in\mathbb{R}^2\cap \mathcal{B}(\underline{0},\delta)\setminus\underline{0}$$ $$\left|\frac{f(x+h_1+h_2)-f(x+h_1)-f(x+h_2)+f(x)}{h_1h_2}-f''(x)\right|<\varepsilon.$$

I'm not $100\%$ sure of this statement (in particular of the fact that the two increments have to independent), but it looks right to me. For sure you need not centered schemes though.

  • 1
    $\begingroup$ The centered difference formula is not a problem: for the first derivative to exist it can't be odd around our point $x_0$ unless it is zero. But you are right that $h$ and $k$ need to be able to vary independently. The counterexample OP cites will fail in this case. $\endgroup$ Jun 19, 2012 at 18:08
  • $\begingroup$ I think that if you use centered schemes, even if $h$ and $k$ are independent, the counterexample that tampis posted still holds (if I did my calculations correctly). But I will double check it. $\endgroup$
    – bartgol
    Jun 19, 2012 at 21:26
  • $\begingroup$ I'm starting to believe that my definition of $f''$ in terms of $f$ is too restrictive. Indeed, if the condition above is satisfied $\forall\underline{h}\in\mathcal{B}(\underline{0},\delta)\setminus\underline{0}$, taking the limit as $h_1$ goes to zero, we have the definition of $f''$ as the limit of the increment quotient of $f'$. So if that definition is satisfied, then we are fine. However, it might be too restrictive. Indeed, $f''$ is defined as the limit of the increment quotient of $f'$, that is when '$h_1$ has been already sent to zero'...in some sense. $\endgroup$
    – bartgol
    Jun 19, 2012 at 21:36
  • 1
    $\begingroup$ The increments $h_1$ and $h_2$ do have to be independent; otherwise you just reduce to $\lim_{h\to 0} (f(x+2h)-2f(x+h)+f(x)/h^2$, which is not sufficient for the second derivative to exist; consider e.g. $f(x) = x^3\sin(1/x)$ (with $f(0)=0$). $\endgroup$ May 13, 2014 at 16:57
  • 1
    $\begingroup$ However, even with independent increments, this definition does not even imply that $f$ is continuous at $x$, as pointed out by Tom Goodwillie in his answer to mathoverflow.net/questions/165704/… --- if $f$ is a $\mathbb{Q}$-linear map $\mathbb{R}\to \mathbb{R}$, then the "second-order difference quotient" is identically zero. $\endgroup$ May 13, 2014 at 16:59

If we are not allowed to talk about $f'(x)$ for $x\ne x_0$ it is not possible to talk about $f''(x_0)$ in the proper sense. One could, however, approach the idea of $f''(x_0)$ via the Taylor expansion of $f$ at $x_0$:

The function $f$, defined in a neighborhood of $x_0$ has second derivative $b$ at $x_0$ if there is an $a\in{\mathbb R}$ such that $$\lim_{h\to 0}{f(x_0+h)-f(x_0)- a h \over h^2}={b\over2}\ .$$ This $\lim$-condition can obviously be expanded into $\epsilon$-$\delta$-language.

Note, however, that the function $f(x):=x^3$ $(x\in{\mathbb Q})$ and $:=0$ $(x\notin{\mathbb Q})$ would have $f''(0)=0$ according to this definition.

  • $\begingroup$ Indeed, this defines what is known as "pointwise second derivative" , which exists more often than $f''(x)$. However, I question why you say "not possible". @bartgol suggested a plausible definition in terms of a limit in $\mathbb R^2$. So far we have no proof that it works, but no counterexample either. $\endgroup$
    – user31373
    Jun 19, 2012 at 16:48
  • 2
    $\begingroup$ @Leonid Kovalev: Do you have any references for the concept of "pointwise second derivative$? A search in google just gave me links to articles which seemed to be really hard to understand for me. $\endgroup$ Jun 22, 2012 at 9:08
  • $\begingroup$ @tampis Definition: $f$ has a pointwise derivative of order $k$ at point $a$ if there exists a polynomial $p$ of degree $k$ such that $\lim_{x\to a}\frac{f(x)-p(x)}{(x-a)^k}=0$. If this holds, we define $f_{pt}^{(k)}(a)=p^{(k)}(a)$. When $k=1$, this is the same notion as $f'(a)$; but for $k>1$ the existence of $f_{pt}^{(k)}(a)$ does not imply the existence of $f^{(k)}(a)$... You will not find this notion in elementary textbooks, because it's not needed there. But there are situations, such as investigation of nonsmooth convex functions in $\mathbb R^n$, when it becomes useful (with $k=2$). $\endgroup$
    – user31373
    Jun 22, 2012 at 14:50
  • $\begingroup$ @user31373 : It seems to me that this definition should explicitly require $f$ to be continuous at $a$; the definition as you state it says nothing at all about $f(a)$, but we cannot recreate the usual notion for $k = 1$ without this. (Then pointwise differentiability is weaker than differentiability for $k > 1$, equivalent for $k = 1$, and stronger for $k = 0$, since even 0-times pointwise differentiability explicitly requires continuity.) $\endgroup$ Oct 14, 2017 at 4:03
  • $\begingroup$ While I'm commenting, I like to think that the $pt$ in $f^{(k)}_{pt}$ stands for Peano–Taylor; the fact that the pointwise derivative equals the derivative when the latter exists is essentially Taylor's theorem with the Peano form of the remainder. (The polynomial $p$ in user31373's definition must be the $k$th-order Taylor polynomial.) $\endgroup$ Oct 14, 2017 at 4:06

In this community wiki post I want to collect all progress we made so far in answering this question. Please feel free to edit it and to extend it with your ideas (you can also start a new community wiki post if you have a new approaches)


Some approaches

  • I know the formula $f''(x) = \lim_{h\rightarrow 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}$ but this limit does not provide the existence of the second derivative (see section “limit” of wikipedia article “second derivative”). So we cannot derive an epsilon-delta definition from the limit $\lim_{h\rightarrow 0} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}$.

  • There is the idea to apply the above epsilon-delta definition to the first derivative:

    $$\forall \epsilon > 0\, \exists \delta > 0\, \forall y \in D\setminus \{x\}:|y-x|<\delta \Rightarrow \left|\frac{f'(y)-f'(x)}{y-x}-f''(x)\right|<\epsilon$$

    But then we already used the existence of the first derivative in our definition. To be a definition we shall conclude from it, that the function is differentiable in a neighborhood of $x$ (This might be hard, I know ;-) ).

  • There is the approach by Christian Blatter to define the second derivative from the taylor series $f(x_0+h)=f(x_0)+f'(x)h+\tfrac 12 f''(x) h^2$, so that it should be the limit

    $$\lim_{h\to 0}2\cdot {f(x_0+h)-f(x_0)- a h \over h^2}=b$$

    whereby $a$ shall be the unique number for which the above limit exists. Unfortunately the function $f(x) := \begin{cases} x^3 & ;x\in\mathbb{Q} \\ 0&;x\notin\mathbb{Q} \end{cases}$ is a function which is not differentiable in a neighborhood of $x$ but the above limit exists for $a=0$. (Notice: the above formula gives a definition for the "pointwise second derivative", please see the comment of Leonid Kovalev)

bartgol's approach

bartgol has the following idea (see his answer):

$$\forall \varepsilon>0, \exists \delta>0 : \forall (h,k)\in\mathbb{R}^2\cap \mathcal{B}(\underline{0},\delta)\setminus\underline{0}$$ $$\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}-f''(x)\right|<\varepsilon.$$

  • Harald Hanche-Olsen mentioned the double mean value theorem in the comments. For me it seems to have a somehow similar form as bartgol's idea.

  • The counterexample $f(x) := \begin{cases} x^3 & ;x\in\mathbb{Q} \\ 0&;x\notin\mathbb{Q} \end{cases}$ does not work for this approach ;-) Let $x=0$, $k+h\in\mathbb{Q}$ with $k,h\notin\mathbb{Q}$. Then we have


    If we fix $h<\delta$ and let $k\rightarrow0$ the amount of $\frac{h^2}{k}$ will get arbitrary high, while $3h+3k+\frac{k^2}{h} \rightarrow 3h$ so that $\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}-f''(x)\right|$ cannot be smaller than $\varepsilon$.

  • I wanted to find a proof that $f$ has to be differentiable in a neighborhood of $x$. Therefore there has to be an open interval $O$ so that for all $x+h\in D\cap O$ we have:

    $$\forall \epsilon > 0\, \exists \delta > 0\, \forall k,\tilde k\in\mathbb{R} : |k| < \delta \land |\tilde k |<\delta \land x+h+k,x+h+\tilde k\in D \Rightarrow \left|{f(x+h+k)-f(x+h) \over k} - {f(x+h+\tilde k)-f(x+h) \over \tilde k}\right|<\epsilon$$

    With the representation $f(k+h+k)=f(x+h)+f(x+k)-f(x)+f''(x)hk+R(h,k)$ ($|R(h,k)|<|\epsilon h k|$) I got

    $$\begin{align}&\left|{f(x+h+k)-f(x+h) \over k} - {f(x+h+\tilde k)-f(x+h) \over \tilde k}\right|\\=&\left|{f(x+k)-f(x)\over k }-{f(x+\tilde k)-f(x)\over \tilde k}+{R(h,k)\over k}-{R(h,\tilde k)\over \tilde k}\right|\end{align}$$

    If one can prove the differentiability of $f$ at $x$ the term ${f(x+k)-f(x)\over k }-{f(x+\tilde k)-f(x)\over \tilde k}$ can be made arbitrary small for $k,\tilde k\rightarrow 0$. Unfortunately we have $\left|{R(h,k)\over k}\right|<|\epsilon h|$so that we have no control over this term if $k$ goes to zero.

  • My interpretation of bartgol's idea for arbitrary functions is (let be $f:D\rightarrow \mathbb{R}$): Forall $\epsilon > 0$, there is a $\delta > 0$, so that for all $x+h+k\in D$ with $|h|+|k|<\delta$ we have

    $$f(x+h+k)=f(x+h)+f(x+k) -f(x)+f''(x)hk+R(h,k)$$

    so that $|R(h,k)|<|\varepsilon h k|$.

    • There is a counterexample for the above interpretation of bartgol's idea (Please notice, that bartgol's idea may still work for functions with open domains): Let $A:=\{q\cdot\pi : q\in\mathbb{Q}\}$ and $B:=\{q\cdot\sqrt{2} : q\in\mathbb{Q}\}$. $A$ and $B$ are abelian groups with $+$ as group operation and $A\cap B=\{0\}$. If we just consider $k,h\in (A \cup B)\setminus\{0\}$ we have $k+h\in A \Rightarrow k\in A \land h \in A$ and $k+h\in B \Rightarrow k\in B \land h \in B$. Now we define $f:A\cup B\rightarrow \mathbb{R}: x \mapsto \begin{cases} x^3 & ;x\in A\setminus\{0\} \\ 0 & ; x\in B\end{cases}$. For $x=0$ we have either:

      $$\begin{align}\left|\frac{f(x+h+k)-f(x+h)-f(x+k)+f(x)}{hk}\right|&=\left|\frac{(h+k)^3-h^3-k^3}{hk}\right|\\&=\left|3h+3k\right|\\&\le 3 (|h|+|k|)\end{align}$$



      This proves that the above statement is true for $f$ with $f''(0)=0$. Because $A$ and $B$ are dense in $\mathbb{R}$ the concept of derivation is well defined for $f$. But because there are noncontinuous jumps in every neighborhood of a point $x\in A \cup B\setminus \{0\}$ the function $f$ is not differentiable in a neighborhood of $x=0$.

  • Actually, this definition does not work for functions defined on open domains either. Consider a $\mathbb{Q}$-linear function $f:\mathbb{R}\to \mathbb{R}$. Then $f(x+h+k)-f(x+h)-f(x+k)+f(x)$ is identically zero, so this definition would give $f''(0)=0$; but such a function need not even be continuous. This was pointed out by Tom Goodwillie here. He did show, however, that this definition does work if we assume that $f$ is differentiable in a neighborhood.


Here is an enhancement of bartgol's idea which also implies that the function is differentiable at $x$. There is a number $a$ such that $\forall \epsilon>0 \;\exists \delta>0$ such that if $h^2 h^k + v^2 <\delta$, then

$$ \Big| f(x+h+k+v) - f(x+h) - f(x+k) + f(x) - c h k - a v \Big| < \epsilon. $$

Letting $v=0$ we recover bartgol's definition. But letting $h=k=0$, we recover the statement that $f$ is differentiable at $x$ with derivative $a$. Therefore, if $f$ satisfies this definition, then it is differentiable at $x$; and furthermore if it is differentiable in some neighborhood of $x$, then it is twice differentiable at $x$ and $f''(x)=c$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .