"Gradient descent is a first-order optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point."

605 questions
Filter by
Sorted by
Tagged with
7 views

26 views

### How to show steepest descent will not converge to local but to global minima of funtion.

for a function $f(x, y) = 0.5*x^2 +0.25*y^4 −0.5*y^2$. Show that there are infinitely many starting points for which gradient (steepest) descent will not converge to a local, let alone global, ...
15 views

### Differentiable approximation of supremum of a function

I am trying to solve an optimization problem of the form: min$_y$ f(y), s.t. g(y,t)$\leq0$ ∀ t∈D. As I can calculate the gradients for f and g, I was thinking of using a penalty method, in which ...
15 views

I saw the gradient of a function, $f(x,y) = \| x - y \|^2$ as $\nabla f_x = 2x^T(x-y)$ where $x$ and $y$ are column vectors. I really would love to know the Mathematics behind this. Also, I thought ...
43 views

My question is very similar to this one and this one, but they haven't been answered. Let $f \in C^2(\mathbb{R}^d, \mathbb{R})$ have compact sublevel sets and isolated critical points, and consider ...
32 views

### Tangent plane parallel to given plane

Let $S$ be the surface in $R^3$ given by $z = \sqrt{2x^2+y^4+1}$ and $P$ be the plane given by $x-y-z=0$. Find an equation of the plane tangent to $S$ and parallel to $P$. Progress I have made so far:...
14 views

### Find standard deviation $s$ such that $\mathbb{P}(X \in [-t, t]) = 1 - \alpha$ for $X \sim \mathcal{N}(0, s)$

Let $t >0$ and $\alpha \in (0, 1)$. I am looking for the standard deviation $s > 0$ such that $\mathbb{P}(X \in [-t, t]) = 1 - \alpha$ for any $X \sim \mathcal{N}(0, s)$. Let $s$ be a ...
32 views