Let $(X,||\cdot||_X)$ be a Banach space and $X^*$ it's dual of linear functionals $X\to\mathbb{R}$. The Fréchet derivative $\nabla f(x)$ of a function $f:X\to\mathbb{R}$ at $x$ is an element of $X^*$.

What exactly does "Lipschitz-continuity of the gradient" mean in this context?

Is it $\forall x,y \in X$ $$ || \nabla f(x) - \nabla f(y)||_{X^*} \leq L || x -y ||_{X} $$ where $|| g ||_{X^*} = \sup \{ |g(x)| : x \in X, ||x|| \leq 1 \}$ is the dual norm and $L$ the Lipschitz constant or

$$ |\langle \nabla f(z), x-y \rangle| \leq L || x -y ||_{X} $$ where $\langle \cdot, \cdot \rangle$ is the duality map?

If the latter is correct, does it have to hold for all $z \in X$?

Edit: An example that would help me to understand this issue better would be $$ f: x \mapsto \frac{1}{2}||x-c||_X^2 $$ for some constant $c \in X$, with derivative $Df(x): h \mapsto \langle x-c,h \rangle$ and gradient $\nabla f(x) = x-c$. How can I calculate the Lipschitz constant?

  • 3
    $\begingroup$ It is the first: $ || \nabla f(x) - \nabla f(y)||_{X^*} \leq L || x -y ||_{X} $ $\endgroup$ – daw Jul 29 '15 at 11:19
  • $\begingroup$ The second is always true - there the constant $L$ is just the dual norm of $\nabla f(z)$ $\endgroup$ – Svetoslav Jul 29 '15 at 12:58

The Fréchet derivative $\nabla f(x)$ of a function $f:X\to\mathbb{R}$ at $x$ is an element of $X^*$.

And therefore, $\nabla f$ is a map from $X$ to $X^*$. The Lipschitz condition makes sense whenever there is a map between metric spaces; and both $X$ and $X^*$ have metrics induced by their norms. Explicitly, $$\| \nabla f(x) - \nabla f(y)\|_{X^*} \leq L \| x -y \|_{X}$$

Your example $$f(x) = \frac{1}{2}\|x-c\|_X^2$$ is very interesting. In a general Banach space, $\nabla f$ is not even defined: for example, the $\ell_1$ norm is not differentiable when one of coordinates is zero (consider two-dimensional model to make this very explicit).

A space for which $f$ is Fréchet differentiable is called a space with Fréchet differentiable norm (the norm itself is of course not differentiable at $0$, but the term refers to differentiability at nonzero points). In lecture 23 here is is proved that in such a space, $\nabla f$ is automatically continuous. The map $\nabla f$ is called the support map or the duality map: it sends each element $x$ to a linear functional $\varphi $ such that $\|\varphi\|=\|x\|$ and $\|\varphi(x)\|=\|x\|^2$.

In general, $\nabla f$ is not even uniformly continuous, let alone Lipschitz continuous.

  • $\nabla f$ is uniformly continuous $\iff$ $X$ is a uniformly smooth space
  • $\nabla f$ is Lipschitz continuous $\iff$ $X$ is a uniformly smooth space with quadratic modulus of smoothness.

The Lipschitz constant of $\nabla f$ depends on the multiplicative constant in the modulus of smoothness.

  • $\begingroup$ Thx. Assume in the example X and f are sufficient "nice". I have a problem calculating $||\nabla f(x) - \nabla f(y)||_{X^*}$ and think it should be $||Df(x) - Df(y)||$ or how do I evaluate $||x-c - y+c||_{X^*}$ for $\{z \in X, ||z|| \leq 1 \}$? $\endgroup$ – Manuel Schmidt Jul 31 '15 at 5:21
  • $\begingroup$ What does $Df$ mean, if not the same as $\nabla f$? How to find the Lipschitz constant of a map depends very much on what that map is. You mentioned the square of the norm... that is still too broad a question, it all depends on how the norm is defined. "How does one find the Lipschitz constant of some map?" by studying it in detail. $\endgroup$ – user147263 Jul 31 '15 at 5:24
  • $\begingroup$ OK, then I got confused by the comment of @Siminore: math.stackexchange.com/questions/1368470/… $\endgroup$ – Manuel Schmidt Jul 31 '15 at 5:28
  • $\begingroup$ Ah, that was about Hilbert spaces. The geometry of the norm in Hilbert spaces is totally trivial compared to what one finds in Banach spaces. $\endgroup$ – user147263 Jul 31 '15 at 5:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.