1
$\begingroup$

Let $(X,||\cdot||_X)$ be a Banach space and $X^*$ it's dual of linear functionals $X\to\mathbb{R}$. The Fréchet derivative $\nabla f(x)$ of a function $f:X\to\mathbb{R}$ at $x$ is an element of $X^*$.

What exactly does "Lipschitz-continuity of the gradient" mean in this context?

Is it $\forall x,y \in X$ $$ || \nabla f(x) - \nabla f(y)||_{X^*} \leq L || x -y ||_{X} $$ where $|| g ||_{X^*} = \sup \{ |g(x)| : x \in X, ||x|| \leq 1 \}$ is the dual norm and $L$ the Lipschitz constant or

$$ |\langle \nabla f(z), x-y \rangle| \leq L || x -y ||_{X} $$ where $\langle \cdot, \cdot \rangle$ is the duality map?

If the latter is correct, does it have to hold for all $z \in X$?

Edit: An example that would help me to understand this issue better would be $$ f: x \mapsto \frac{1}{2}||x-c||_X^2 $$ for some constant $c \in X$, with derivative $Df(x): h \mapsto \langle x-c,h \rangle$ and gradient $\nabla f(x) = x-c$. How can I calculate the Lipschitz constant?

$\endgroup$
2
  • 3
    $\begingroup$ It is the first: $ || \nabla f(x) - \nabla f(y)||_{X^*} \leq L || x -y ||_{X} $ $\endgroup$
    – daw
    Commented Jul 29, 2015 at 11:19
  • $\begingroup$ The second is always true - there the constant $L$ is just the dual norm of $\nabla f(z)$ $\endgroup$
    – Svetoslav
    Commented Jul 29, 2015 at 12:58

1 Answer 1

2
$\begingroup$

The Fréchet derivative $\nabla f(x)$ of a function $f:X\to\mathbb{R}$ at $x$ is an element of $X^*$.

And therefore, $\nabla f$ is a map from $X$ to $X^*$. The Lipschitz condition makes sense whenever there is a map between metric spaces; and both $X$ and $X^*$ have metrics induced by their norms. Explicitly, $$\| \nabla f(x) - \nabla f(y)\|_{X^*} \leq L \| x -y \|_{X}$$

Your example $$f(x) = \frac{1}{2}\|x-c\|_X^2$$ is very interesting. In a general Banach space, $\nabla f$ is not even defined: for example, the $\ell_1$ norm is not differentiable when one of coordinates is zero (consider two-dimensional model to make this very explicit).

A space for which $f$ is Fréchet differentiable is called a space with Fréchet differentiable norm (the norm itself is of course not differentiable at $0$, but the term refers to differentiability at nonzero points). In lecture 23 here is is proved that in such a space, $\nabla f$ is automatically continuous. The map $\nabla f$ is called the support map or the duality map: it sends each element $x$ to a linear functional $\varphi $ such that $\|\varphi\|=\|x\|$ and $\|\varphi(x)\|=\|x\|^2$.

In general, $\nabla f$ is not even uniformly continuous, let alone Lipschitz continuous.

  • $\nabla f$ is uniformly continuous $\iff$ $X$ is a uniformly smooth space
  • $\nabla f$ is Lipschitz continuous $\iff$ $X$ is a uniformly smooth space with quadratic modulus of smoothness.

The Lipschitz constant of $\nabla f$ depends on the multiplicative constant in the modulus of smoothness.

$\endgroup$
4
  • $\begingroup$ Thx. Assume in the example X and f are sufficient "nice". I have a problem calculating $||\nabla f(x) - \nabla f(y)||_{X^*}$ and think it should be $||Df(x) - Df(y)||$ or how do I evaluate $||x-c - y+c||_{X^*}$ for $\{z \in X, ||z|| \leq 1 \}$? $\endgroup$ Commented Jul 31, 2015 at 5:21
  • $\begingroup$ What does $Df$ mean, if not the same as $\nabla f$? How to find the Lipschitz constant of a map depends very much on what that map is. You mentioned the square of the norm... that is still too broad a question, it all depends on how the norm is defined. "How does one find the Lipschitz constant of some map?" by studying it in detail. $\endgroup$
    – user147263
    Commented Jul 31, 2015 at 5:24
  • $\begingroup$ OK, then I got confused by the comment of @Siminore: math.stackexchange.com/questions/1368470/… $\endgroup$ Commented Jul 31, 2015 at 5:28
  • $\begingroup$ Ah, that was about Hilbert spaces. The geometry of the norm in Hilbert spaces is totally trivial compared to what one finds in Banach spaces. $\endgroup$
    – user147263
    Commented Jul 31, 2015 at 5:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .