# If $f$ is uniformly differentiable then $f '$ is uniformly continuous?

The following theorem is true?

Theorem. Let $U\subset \mathbb{R}^m$ (open set) and $f:U\longrightarrow \mathbb{R}^n$ a differentiable function.

If $f$ is uniformly differentiable $\Longrightarrow$ $f':U\longrightarrow \mathcal{L}(\mathbb{R}^m,\mathbb{R}^n)$ is uniformly continuous.

Note that $f$ is uniformly differentiable if

$\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{[x,x+h]\subset U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!|$ (edited)

$\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{x,x+h\in U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!|\qquad \checkmark$

Any hints would be appreciated.

• What is your definition for uniformly differentiable? – Tomás Sep 5 '13 at 19:49
• @Tomás $\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,[x,x+h]\subset U \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!|$ – felipeuni Sep 5 '13 at 20:02
• What did you try ? (I think you know the proof in the case $\Bbb R \rightarrow \Bbb R$). – Tony Piccolo Sep 6 '13 at 10:26
• Do you know the proof in this case @TonyPiccolo? – Tomás Sep 6 '13 at 12:22
• See my new attempt. – Tony Piccolo Sep 24 '13 at 9:29

Let's build off of Tomas' last remark, slightly modified:

Let $t>0$ be small. Then \begin{eqnarray} \|f'(x)-f'(y)\| &=& \frac{1}{t}\sup_{\|w\|=1}\|\langle f'(x)-f'(y),tw\rangle\| \nonumber \\ &\leq& \frac{1}{t}\sup_{\|w\|=1}\|f(x+tw)-f(x)-[f(y+tw)-f(y)]\| + 2\epsilon \nonumber \end{eqnarray}

It suffices to show that this weighted combination of four close points on a parallelogram can be bounded by $C\epsilon t$.

Let us bound $\|f(x+h) - f(x) + f(x+k) - f(x+h+k)\|_2 \leq C\epsilon(\|h\|+\|k\|)$, and then in this case $\|h\|=t$ and $\|k\|\leq \delta$, so if $t=\delta$ the whole expression is bounded by a constant times $\epsilon$.

Note applying uniform differentiability three times in directions $h,k,$ and $h+k$, for small $\|h\|,\|k\|$ we have

\begin{eqnarray*} \|f(x+h) - f(x) + f(x+k) - f(x+h+k)\| &\leq& \|f'(x)h + f'(x)k - f'(x)(h+k)\|_2 + 3\epsilon(\|h\|+\|k\|)\\ &=& 3\epsilon(\|h\|+\|k\|) \end{eqnarray*}

• Shouldn't the first occurrence of $2 \epsilon$ actually be $2\epsilon/t$? I'm not sure you can magick away the magnitude of $w$ like that. – Anthony Carapetis Sep 10 '13 at 1:17
• @AnthonyCarapetis The original equation had $\epsilon |w|$, but since I replaced $w$ with $tw$, that's where the extra $t$ comes from. – Evan Sep 10 '13 at 1:21
• oh right. It works. – Anthony Carapetis Sep 10 '13 at 1:43
• @Evan only a question, the following equivalence is true ? $\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{[x,x+h]\subset U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!\!|<\epsilon |\!|h|\!|\Longleftrightarrow \forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{x,x+h \in U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!\!|<\epsilon |\!|h|\!|$ – felipeuni Sep 10 '13 at 5:35
• @felipeuni Yes that is true, because the property only depends on $x$ and $x+h$, so there's no distinction here. – Evan Sep 10 '13 at 5:46

This answer only the case $f:U\subset\mathbb{R}\to \mathbb{R}^n$. By hypothesis we have that: given $\epsilon>0$, there exist $\delta>0$ such that if $|x-y|<\delta$ $$\left|\frac{f(x)-f(y)}{x-y}-f'(y)\right|<\frac{\epsilon}{2}\tag{1}$$

and

$$\left|\frac{f(y)-f(x)}{y-x}-f'(x)\right|<\frac{\epsilon}{2}\tag{2}$$

From $(2)$ we get that $$\left|\frac{f(x)-f(y)}{x-y}-f'(x)\right|<\frac{\epsilon}{2}\tag{3}$$

From $(1)$ and $(3)$ $$|f'(x)-f'(y)|<\epsilon$$

I dont know how to tackle the case $U\subset \mathbb{R}^m$.

Update: I would like to add here some thoughts, maybe it can help someone to give a full answer. First note that

\begin{eqnarray} \|f'(x)-f'(y)\| &=& \sup_{\|w\|=1}\|\langle f'(x)-f'(y),w\rangle\| \nonumber \\ &=& \sup_{\|w\|=1}\|f(x+w)-f(x)-[f(y+w)-f(y)]+o(x,w)-o(y,w)\| \nonumber \end{eqnarray}

By hypothesis, $\|o(x,w)-o(y,w)\|\leq \epsilon\|w\|+\epsilon\|w\|$, hence for $\|w\|=1$ we have that $\sup_{\|w\|=1}\|o(x,w)-o(y,w)\|$ is small, independetly of $x,y$. Therefore, it remains to prove that $f$ is uniformly continuous. One way to prove it is for example showing that $f'$ is bounded, which I think is true when $U$ is bounded. When $U$ is unbounded, I think we need a direct argument to show that $f$ is uniformly continuous.

• [Tomás] Note that $[x,y]\subset \mathbb{U}$ then $U$ should be an interval, definition of uniformly differentiable. – felipeuni Sep 9 '13 at 6:36
• @felipeuni you need to specify where are $x,y$. If $x,y$ are any points on $U$, then the only thing you can conclude is that $U$ is convex. – Tomás Sep 9 '13 at 12:32
• For bounded regions, take the closure to get a compact set, and then it's clear that $f$ is uniformly continuous. For unbounded regions, as you say we need a direct argument, as $f$ is not uniformly continuous in general, with a 1-d counterexample $f(x) = x^2$... – Evan Sep 9 '13 at 23:20

The difficulty here is that the obvious estimate (as in Tomás' answer) only gives a result along lines - i.e. it can't say anything about how $f'(x)h$ changes as you move in any direction but $h$. Thus I think we need some kind of "polarization argument" to get an isotropic result. When $f$ is $C^2$, this is literally polarizing the Hessian matrix $H(x) = f''(x)$ to show it is bounded from the fact that it is bounded on the diagonal. While we cannot use this literally for general $f$, hopefully we can use a finite-difference analogy and the analysis will go through. Here's the proof for the $C^2$ case and an optimistic starting point for the general case.

For any $\epsilon>0$ there is a $\delta>0$ such that for all $x,h$ with $|h|<\delta$, we have (cf. Tomás' answer)

\begin{align} |f(x+h) - f(x) - f'(x)h| &< \epsilon |h| \\ |f(x) - f(x+h) + f'(x+h)h| &< \epsilon |h| \end{align}

which combine to give $$t|(f'(x+t v)-f'(x))v| < 2 \epsilon$$ for $v$ a unit vector, $t<\delta$.

In the $C^2$ case the left side is $t^2 |H(x)(v,v)| + o(t^2)$, so we have a uniform bound $$|H(x)(v,v)| < M \tag{1}.$$ The polarization formula is $$H(x)(v,w) = \frac12 \left( H(x)(v+w,v+w) - H(x)(v,v) - H(x)(w,w) \right). \tag{2}$$

We want to show uniform continuity of $f'$. A line integral estimate gives $$|f'(x)w-f'(y)w| \le |x-y| \sup_{z \in U} |H(z)\left(v,w\right)|$$ where $|x-y|v = x-y$; so it suffices to show that $\sup_{z \in U, |v| = 1} H(z)(v,w)$ is finite. Applying $(1)$ and $(2)$ we estimate

\begin{align} |H(z)(v,w)| & \le \frac12 M \left( |v+w|^2 + |v|^2 + |w|^2 \right)\\ & \le M \left( 1 + |w| + |w|^2 \right) \end{align}

so we are done.

Now to try generalising this. Attempting to write down the polarization formula using finite difference terms gives

\begin{align} &(f'(x+v+w) - f'(x+v))w + (f'(x+w+v)-f'(x+w))v \\ = &(f'(x+v+w) - f'(x))(v+w) - (f'(x+v)-f'(x))v-(f'(x+w) - f'(x))w \\ \end{align}

which (since $f$ is uniformly differentiable) is bounded by $2 \epsilon (|v+w| + |v| + |w|)$.

Now we would like to equate the two terms on the LHS, which is the reason polarization works for symmetric forms. Our expression is of course not symmetric but we can hope it is close enough for small $u,v$ - I have no idea whether or not this can be done without more assumptions, however. If anyone has any ideas (or has spotted any mistakes) please let me know.

• Maybe we could approximate $f$ by $C^2$ functions? – Tomás Sep 9 '13 at 14:20
• @AnthonyCarapetis@Tomás I think the following article Theorem $3.4$ could help. physics.umanitoba.ca/~khodr/Publications/RSImplicitFT-Final.pdf – felipeuni Sep 9 '13 at 22:43
• @felipeuni: that theorem is for non-Archimedian fields, however at first glance the proof seems to work for the Euclidean case too! In fact it may even directly prove what you're looking for - I haven't been through it in detail but the estimates are probably all uniform in $x_0$. I think it's very similar to Evan's answer, which seems to be a complete proof. – Anthony Carapetis Sep 10 '13 at 2:00
• @AnthonyCarapetis the link is broken. The name of the article is "The implicit function theorem in a non-Archimedean setting". – Guillaume F. Jul 18 '18 at 17:07

Okay, let's get rid of the $R'$ and try the same thing this way. f is differentiable at $X_0$ means that $\exists$ a g which is linear throughout some neighborhood U of $X_0$, g(X) = $f(X_0)$ + (M,$(X-X_0)$ [ where the second term is the inner product of $(X-X_0)$ with a constant vector M] and for which $\lim_{X \rightarrow X_0}\frac{f(X) -g(X)}{X - X_0}$ = 0. This means that f is practically linear throughout U.

Now if f were precisely linear we would have no problem, since $f(X) - f(X_0)$ = (M,$X-X_0)$ and this is uniformly continuous throughout U. The problem is that f is not exactly linear. I am using R(X) to represent the difference between f, the not quite linear, and g the exactly linear.

So the question of uniform continuity of f in the neighborhood U comes down to the behavior of R(X) in U. What do we know about R? We know that $\lim_{X \rightarrow X_0} \frac{R(X)}{X-X_0}$ = 0 and further that the convergence is uniform in U. That means that if you pick an $\epsilon$, there is a $\delta$ independent of X such that in $V \subset U$ a neighborhood of $X_0$ of radius $\delta$, if $X_1$ and $X_2$ are in V then $\frac {|R(X_1) - R(X_2)|}{|X_1-X_2|}$ < $\epsilon$. Substituting back the definition of R we have

$R(X_1)$ = $f(X_1)-g(X_1)$ = $f(X_1) - f(X_0)- (M,(X_1-X_0))$ and similarly for $X_2$. So

$\frac {|R(X_1) - R(X_2)|}{|X1-X_2|}$ = $\frac{f(X_1) -f(X_2) - [(M,(X_1-X_0))-(M,(X_2-X_0))]}{X_1 - X_2}$ < $\epsilon$ when $X_1, X_2$ are in V.

We get $|f(X_1)-f(X_2)|$ < $\epsilon|(X_1-X_2)|$ + $[(M,(X_1-X_0))-(M,(X_2-X_0))]$. The terms in square brackets go to 0 uniformly as $X_1 \rightarrow X_2$ because they are the difference of a linear function at 2 points in V, and the $\epsilon|(X_1-X_2)|$ likewise .

The proof of theorem $3.4$ in your reference contains the idea to solve the problem: one has to prove the continuity of partial derivatives.
The reasoning is valid apart from the non-Archimedean and local context of the writing.

In the $\varepsilon-\delta \,$ game let $\delta \,$ be a reply to $\varepsilon /4$ as for the differentiability of $\mathbf f$ and let $\delta_1$ be a reply to $\varepsilon \delta/4$ as for the continuity of $\mathbf f$.

In the following $f_j^i$ is the derivative of $f^i$ respect to $x_j \,$.

Now choose $\mathbf x,\mathbf y \in U$ such that $||\mathbf x-\mathbf y||<\min \{\delta,\delta_1\}$.
Then $$|f_j^i(\mathbf x)\delta-f_j^i(\mathbf y)\delta| \le$$$$|f^i(\mathbf x+\delta\hat {\mathbf e}_j)-f^i(\mathbf x)-f_j^i(\mathbf x)\delta|$$$$+ \,|f^i(\mathbf y+\delta\hat {\mathbf e}_j)-f^i(\mathbf y)-f_j^i(\mathbf y)\delta|$$$$+ \,|f^i(\mathbf x)-f^i(\mathbf y)|+|f^i(\mathbf x+\delta\hat {\mathbf e}_j)-f^i(\mathbf y+\delta\hat {\mathbf e}_j)|$$$$< \varepsilon \delta$$from which the continuity of $f_j^i \,$.
The continuity of the (total) derivative follows from $$||\mathbf f'(\mathbf x)-\mathbf f'(\mathbf y)|| \le \left \{\sum_{i,j} \;[f_j^i(\mathbf x)-f_j^i(\mathbf y)]^2 \right \} ^\frac 12$$by the Schwarz inequality.

Edit:$\;$ OP advises that the proof doesn't work: in fact the uniform continuity of $\mathbf f$ is not assured because $U$ is open.

new attempt

Let $U$ be simply open: I want to prove that $f'$ is uniformly continuous on the set$$U_{\Delta}=\{x\in U:\ {\rm dist}\,(x,\Bbb R^m\backslash U)>\Delta\}$$with $\Delta>0$ chosen sufficiently small.

Avoiding some detail, let $x \in U_{\Delta}$ and let $\delta>0$ corresponding to $\varepsilon>0$ and such that $\delta<\Delta$.

We have $$(f'(x+h)-f'(x))(k)=$$$$-[f(x+k)-f(x+h)-f'(x+h)(k-h)]+$$$$f(x+k)-f(x)-f'(x)(k)+$$$$f(x)-f(x+h)-f'(x+h)(-h)$$Take $h$ and $k$ so that $\|h\|<\frac \delta 2$ and $\|k\|=\|h\|\,$: hence $x+h,x+k \in U$.

Then $$\|(f'(x+h)-f'(x))(k)\|<4\varepsilon \|k\|$$

Now let $u \in \mathbb R^m$ such that $\|u\|=1$ and let $k=\|h\|\,u\,$: we have $$\|(f'(x+h)-f'(x))(u)\|<4\varepsilon$$Then $$\|f'(x+h)-f'(x)\|\le4\varepsilon$$

• [@TonyPiccolo] Then $\displaystyle \delta_1=\delta_1(\frac{\epsilon \delta}{4},x)$ as $f$ is continuous only, $\delta_2=\min \{\delta,\delta_1\}$ should not depend on the points $x,y.$ – felipeuni Sep 13 '13 at 16:43
• The point is that $\mathbf f$ is uniformly continuous because it is uniformly differentiable: modify slightly the statement and the proof of proposition $3.3$ in your reference and you have the result. – Tony Piccolo Sep 13 '13 at 17:17
• [@TonyPiccolo] Note that $f:\mathbb{R}\longrightarrow \mathbb{R},f(x)=x^2$ is uniformly differentiable but $f$ is non-uniformly continuous. – felipeuni Sep 13 '13 at 18:03
• Sorry, but I had forgotten that $U$ is open. – Tony Piccolo Sep 13 '13 at 22:37
• However do you think that proving the uniform continuity of partial derivatives is possible ? For me it should be the most natural way to solve your question. – Tony Piccolo Sep 15 '13 at 6:49

From the uniform differentiabilty, we can find a $t>0$ such that, for every ${\bf v}$ with $\|{\bf v}\| \le 2t$ and ${\bf x} \in U$, ${\bf x} + {\bf v} \in U$,

$$t^{-1}\| f({\bf x}) - f({\bf x}+{\bf v}) + f'({\bf x}){\bf v}\| < \frac{\epsilon}{5}$$

Now, let take any ${\bf x}$, ${\bf y}$ such that $\|{\bf x} - {\bf y}\| < t$, and ${\bf x}, {\bf y} \in U$. Define ${\bf k}$ = ${\bf y} - {\bf x}$.

Note that for any $t{\bf w}$ with $\| {\bf w} \|=1$, we have $\|t{\bf w} + {\bf k}\| < 2t$.

We then have, for every ${\bf x}$ such that an open ball of radius $2t$ is contained in $U$,

\begin{align} \left\| f'({\bf x}) - f'({\bf y})\right\| &= t^{-1}\max_{\|{\bf w}\|=1}\left\| f'({\bf x}){t\bf w} - f'({\bf x} +{\bf k})t{\bf w}\right\| \\ &\le t^{-1}\max_{|{\bf w}|=1} \left\| ( f({\bf x}+t{\bf w}) - f({\bf x}) + f({\bf x} + {\bf k}) - f({\bf x} + {\bf k} + t{\bf w}) \right\| + \frac{2}{5}\epsilon \\ &\le t^{-1}\max_{|{\bf w}|=1}\left\| f'({\bf x})({t\bf w} + {\bf h} -t{\bf w} -{\bf h})\right\| + \epsilon\\ &= \epsilon \end{align}

However, I do not know how to prove it for ${\bf x}$ less than $2t$ away from a point outside $U$.

Maybe some sort of extension theorem would help?

Otherwise, we can strengthen the definition of uniform differentiability to the following:

$\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{x \in U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!|\qquad$

in which case it does work.

Let us start with the simplest case f:$R^1 \rightarrow R^1$. The discussion below will generalize nicely to more dimensions.

Then f is differentiable at $x_0$ iff $\lim_{x \rightarrow x_0} {\frac{f(x)-f(x_0)}{x-x_0}}$ exists and is finite. We call that limit $f'(x_0)$ . So if f is differentiable at $x_0$ then we may write

f(x) = $f(x_0) + f'(x_0)(x-x_0) + R(x)$ where $\lim_{x \rightarrow x_0} R(x)/(x-x_0)$ = 0.     (1)

If f is uniformly differentiable that means

$\lim_{x \rightarrow x_0} R(x)/(x-x_0)$ converges uniformly to zero.     (2)

From (1) we have $f'(x)$ = $f'(x_0)$ + R'(x). [We know R is differentiable because it is the difference between f which is differentiable and a linear polynomial.]

So $\lim_{x \rightarrow x_0} (f'(x)-f'(x_0))$ = $\lim_{x \rightarrow x_0} R'(x)$ = $\lim_{x \rightarrow x_0} R(x)/(x-x_0)$ = 0. And this convergence is uniform as per our hypothesis as stated in (2).

Moving on to the case where $f:R^n \rightarrow R^m$. For this part of the discussion capital letters will be used to represent vectors (in whatever dimension).

According to many sources (and I will state specifically Robert Sealey "Calculus of Several Variables") f is differentiable at $X_0$ iff there exists a linear function g (X) = $f(X_0)$ + $(M,X-X_0)$ [where M is a constant vector and (a,b) means the inner product], such that

$\lim_{X \rightarrow X_0} {\frac{f(X)-g(X)}{|X-X_0|}}$ = 0.   (3)

If f is uniformly differentiable that means simply that this limit converges uniformly.We could have stated differentiability this way in the $R^1 \rightarrow R^1$ case, but I stated it as I did in that case to throw some light on what g actually is.

Let R(X) = f(X) - g(X) $\Rightarrow$ f(X) = g(X) + R(X). Since R = f - g then $R/|x-x_0| \rightarrow 0$ uniformly. Notice also that $R(X_0)$ = 0, since it is $f(X_0) - g(X_0)$ and by (3) this must be 0. Since g is linear and f is differentiable R is also differentiable and from the above $R'(X_0)$ = 0.

Looking at $f'$ we have $f'(X) = g'(X) + R'(X) = M + R'(X)$;
$f'(X_0) = M + R'(X_0)$ and thus $f'(X_0) = M$.

Finally $\lim_{X \rightarrow X_0} (f'(X)-f'(X_0))$ = $\lim_{X \rightarrow X_0} R'(X)$ = $\lim_{X \rightarrow X_0}{\frac{R(X)-R(X_0)}{X-X_0}}$ = $\lim_{X \rightarrow X_0}{\frac{R(X)}{X-X_0}}$ and this goes to 0 uniformly because f is uniformly differentiable, R = f - g, and $R(X_0)$ is 0.

• Haven't you assumed $R'$ is continuous? Can we deduce continuously differentiable from uniformly differentiable? – Anthony Carapetis Sep 9 '13 at 6:09
• @BettyMock Note that $f$ is uniformly differentiable if $\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{[x,x+h]\subset U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!\!|<\epsilon |\!|h|\!|$ – felipeuni Sep 9 '13 at 6:15
• In fact I think that for your convergence to be uniform you have used that $R'$ is uniformly continuous when you replaced $\lim_{X \to X_0} R'(X)$ with $R'(X_0)$, so the argument seems circular. – Anthony Carapetis Sep 9 '13 at 6:28
• @AnthonyCarapetis -- The R(x)/$(X-X_0) \rightarrow$ 0 is exactly equation (3), which is the definition of differentiable. f is uniformly differentiable is equivalent to (3) converges uniformly. Re $R'(X_0)$ = 0 we have $(R(X)-R(X_0))/|X-X_0|$ = $R(X)/|X-X_0|$; this converges to 0 uniformly because it is equation (3) where convergence is uniform. And of course this limit is $R'(X_0)$. So I don't think I'm going in a circle. I relied strictly on the definition of differentiable and the given uniform convergence. I don't think I am assuming anywhere that R' is continuous. – Betty Mock Sep 10 '13 at 0:46
• @felipeuni. That is a correct definition of uniformly differentiable. However, every definition can be stated in different ways. I am stating it as equation (3) converges uniformly. If you check it out, you will see they are equivalent. – Betty Mock Sep 10 '13 at 0:49