This tag is for questions relating to subgradient, an iterative method for solving convex minimization problems, used predominantly in Nondifferentiable optimization for functions that are convex but nondifferentiable. The subgradient method is a very simple algorithm for minimizing convex nondifferentiable functions where newton's method and simple linear programming will not work.

227 questions
Filter by
Sorted by
Tagged with
21 views

### Subdifferentials of a strictly convex function are disjoint

I'm trying to prove below result which is mentioned in page 24 of Santambrogio's Optimal Transport for Applied Mathematicians: Calculus of Variations, PDEs, and Modeling. Could you have a check on my ...
40 views

### Are subgradients bounded on compact subsets of an open set?

Context & Motivation: This is a preliminary question that I wanted to answer in order to generalize the nonconvex projected gradient descent method in Chapter 3 of the book Non-convex Optimization ...
1 vote
16 views

### Convergence Rate of Stochastic Gradient Method analysis with more realistic stochastic graident bound

I am reading this Lectur Slides of SGD convergence analysis. In page 3, it says the realistic bound "Just get some extra terms in the result." I tried to use this bound and derive the result,...
1 vote
17 views

### Analyzing the subgradient method with convex and Lipschitz condition

For a convex function $f: D \rightarrow \mathbb{R}$, where $D \subseteq \mathbb{R}^{n}$ is convex and open, define a subgradient of $f$ at $x_{0} \in D$ to be any vector $s \in \mathbb{R}^{n}$ such ...
1 vote
38 views

### Defining the Subgradient with Polar of the Tagent Cone on the Epigraph, or the Convex Hull

To start, I would like to state both definitions for sub-gradient and where they came from. Definition 1 Let $f$ be $\mathbb E \mapsto \mathbb {\bar R}$ with $x\in \text{dom}(f)$ then $v\in \mathbb E$...
1 vote
25 views

### Subgradient method for nonconvex nonsmooth function

Gradient descent or stochastic gradient descent are frequently used to find stationary points (and in some cases even to local minimum) of a nonconvex function. I was wondering if the same can be said ...
32 views

31 views

### Existence of accelerated subgradient methods

Heavy Ball method and Nesterov's gradient method are two kinds of accelerated versions of gradient methods that achieve optimal convergence for smooth optimization. I wonder whether there is an ...
79 views

### If a function is convex, then its subgradient set is non-empty

Theorem: A function $f: \textbf{dom}(f) \to \mathbb R$ is convex if and only if $\textbf{dom}(f)$ is convex and $\partial f(\textbf x)$ is not empty for all $\textbf x \in \textbf{dom}(f)$ I know the ...
1 vote
57 views

### Convex and Lipschitz implies "bounded" subgradient in Banach spaces

In this question: Lipschitz implies bounded gradient it is shown that if $f: \mathbb{R}^n \to \mathbb{R}^n$ is convex and L-Lipschitz the gradient is bounded by L. I wonder if this holds in Banach ...
39 views

1 vote
315 views

I'm trying to calculate the subdifferential of the infinity norm, but I'm a bit stuck. The definition of the subdifferential $\partial f(x)$ of a function $f(x)$ that I am working with is $$\partial f(... 0 votes 1 answer 64 views ### How can I understand the "scaled subdifferential"? The definitions of subdifferential and scaled subdifferential are (screenshot) Subdifferential: The subdifferential of f at x is the set of vectors \partial f(\boldsymbol{x})=\left\{\boldsymbol{... 1 vote 1 answer 501 views ### Find the subdifferential for \max\left(x^2,|x|\right) We define the function f:\mathbb R \rightarrow \mathbb R as the following:$$f\left(x\right)=\max\left(x^2,|x|\right)$$Find the subdifferential \partial f\left(x\right) for all x\in \mathbb R. ... 1 vote 0 answers 167 views ### Calculate subdifferential of a function Calculate the subdifferential of a function f: \mathbb{R}^3 \to \mathbb{R} defined by$$f(x) = \vert x_1 - x_2\vert + \vert x_2 - x_3\vert$$My attempt Applying the sum rule we have$$\partial f(x) ...
Suppose $f(X)$ is a function from $\mathbb{R}^{m\times n}\mapsto \mathbb{R}$ defined by $f(X)=\lVert\sigma(X)\rVert_F^2$, where $\sigma(X)=\max\{X,0\}$ is an entry-wise ReLU function, i.e., mapping ...