"Gradient descent is a first-order optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point."

880 questions
Filter by
Sorted by
Tagged with
1 vote
49 views

### why gradient descent does not always land at the global minimum closest to the starting point?

I am given this function $\boxed{f(x,y)=((x^2+y^2)-1)^2}$. I need to do gradient descent analysis on it. I have studied that it's not trivial to show mathematically "ball reaches to the global ...
• 323
11 views

### Gradient calculation for an autoregressive probit model

I am using the following specification to estimate a binary choice Probit model: $$P(y_t=1|x_t) = \Phi(\pi_t),$$ where $\Phi$ is the cumulative distribution function of the normal distribution. My ...
15 views

### Taking multiple optimization steps on the same trajectory not well justified.

I am reading the paper Proximal Policy Optimization Algorithms found at https://arxiv.org/pdf/1707.06347.pdf. In this paper they say "While it is appealing to perform multiple steps of ...
• 21
1 vote
15 views

43 views

• 448
32 views

### least squares with L1 regularization in selected entries

Say for $x \in \mathbb{R}^n$, I'm minimizing $\|Ax - b \|_2^2$ with L1 regularization on selected entries of $x$. i.e. instead of directly add a $\|x\|_1$ regularization term, it would be on \$|x_i| + |...
• 155