0
$\begingroup$

Looking at the definition of the subdifferential, we have that $v$ is a subdifferential of a function $f$ at a point $x$ if

$$ f(y) \geq f(x) + g^T(y-x), \forall y$$

Now, for $f(x) = \|x\|_2$, it's not clear to me how the result is derived for when $x\neq 0$, which is that $\partial f(x) = \frac{x}{\|x\|_2}$.

$\endgroup$

1 Answer 1

1
$\begingroup$

Before we answer your question, let me be a bit more rigorous with the definitions: Let $f:\mathbb{R}^n \to \mathbb{R}$ be a proper convex function. Then, $g\in \mathbb{R}^n$ is said to be a subgradient of $f$ at $x$ if $$ f(y) \ge f(x) + g^T(y-x) $$ for all $y\in \mathbb{R}^n$. The subdifferential of $f$ at $x$ is, instead, the set of all its subgradients at that point: $$ \partial f(x) = \bigl\{g\in \mathbb{R}^n : g \text{ is a subgradient of $f$ at $x$}\bigr\}. $$ Note the difference between subgradient and subdifferential. Also, note that these definitions hold only for functions that are convex (and proper, but this is only a technicality to avoid some degenerate cases).

Now, if $f$ is differentiable at $x$, then the gradient $\nabla f(x)$ is the only subgradient of $f$ at $x$ and the subdifferential is a singleton: $$ \partial f(x) = \bigl\{\nabla f(x)\bigr\}. $$ The fact that the gradient is a subgradient is a result of $f$ being convex. To show that the gradient, if it exists, is the unique subgradient, you have to look at what happens in the limit for $y\to x$.

Wrapping up, for the $\ell_2$ norm we have (see also https://math.stackexchange.com/a/4335217/1218593) $$ \partial f(x) = \begin{cases} \bigl\{g \in \mathbb{R}^n : \lVert g\rVert_2 \le 1\bigr\} &\text{if } x=0 \\ \bigl\{\nabla f(x)=\frac{1}{\lVert x\rVert_2}x\bigr\} &\text{if } x\ne 0. \end{cases} $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .