0
$\begingroup$

Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be a vector-valued function given by $f(x) = \sqrt{1 + \|x\|^2_2}$. Show that the gradient of $f$ is Lipschitz continuous, in particular with Lipschitz constant $L = 1$.

I want to show this directly from $\|\nabla f(x) - \nabla f(y)\|_2 \leq \|x - y\|_2 \; \forall x, y \in \mathbb{R}^n$. However, the gradient of $f$ is not exactly very clean, and I get lost in the algebra. Is there a simpler way to do this that I'm missing?

$\endgroup$
  • $\begingroup$ Maybe a direct computation of the gradient and looking for an upper bound... $\endgroup$ – energy Feb 14 at 2:52
0
$\begingroup$

The gradient of your function is $$ \nabla f(x)=\frac {2x}{2\sqrt{1+\|x\|^2}}=\frac {x}{\sqrt{1+\|x\|^2}} $$ whose norm is bounded by $1$, $$ \|\nabla f(x)\|= \frac {\|x\|}{\sqrt{1+\|x\|^2}} \le 1 $$ Therefore, your function is Lipschitz continuous with Lipschitz constant equal $1$ (just an application of the mean value theorem between any two points in $\mathbb{R}^n$).

$\endgroup$
  • $\begingroup$ Ah, silly. I was too stuck on the strict definition to realize I could simply bound the function in this fashion. Thank you. $\endgroup$ – rw435 Feb 14 at 3:13
  • $\begingroup$ @rw435 Does this answer your question? This shows $f$ is Lipschitz, not $\nabla f$ as the question you wrote suggested. Or was it a typo, and Lipschitzness of $f$ was all you wanted? $\endgroup$ – user744868 Feb 14 at 3:20
  • $\begingroup$ You are right @user744868 I thought it was the Lipschitz continuity of $f$ itself. I will edit my answer $\endgroup$ – GReyes Feb 14 at 3:27
  • $\begingroup$ Hmmm indeed. To prove $L$-Lipschitz continuity of $\nabla f$, do we not actually need to prove $||\nabla ^2 f|| \leq L$? I overlooked this... $\endgroup$ – rw435 2 days ago
  • $\begingroup$ I think this exchanges my original struggle of rearranging the terms in the gradient difference to computing the Hessian, which is arguably even less preferable $\endgroup$ – rw435 2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.