"Gradient descent is a first-order optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point."

999 questions
Filter by
Sorted by
Tagged with
23 views

I am working on a non-linear optimization problem, containing bounds and constraints for the variables. Very complex problem involving networks and logical functions. I have been switching a tool from ...
18 views

### How to calculate convergence rate of gradient descent

I am researching on gradient descent. I am looking at the convex case with Lipschitz-continous gradients. For that I'm using Nesterov's "Lectures on convex optimitzation". His result for the ...
1 vote
44 views

Background: Regular gradient descent can be written something like $x_{t + 1} = x_t - \eta g_t$, where $g_t$ is the gradient of the function we're trying to optimize. Problem: If we have a (symmetric, ...
I need to minimize sum of modulus of linear combinations of the variables. For example: $$f = |1+a-2b+3c| + |-a+b+c| + |a| + |b| + |c|$$ My initial plan was to calculate gradient and then perform ...