"Gradient descent is a first-order optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point."

1,041 questions
Filter by
Sorted by
Tagged with
15k views

1 vote
35 views

8 views

1 vote
22 views

### How to comment on goodness of loss functions?

I have two loss functions $\mathcal{L}_1$ and $\mathcal{L}_2$ to train my model. The model is predominantly a classification model. Both $\mathcal{L}_1$ and $\mathcal{L}_2$ takes two variants of the ...
36 views

### Gradient descent for solving complex-valued $Ax = b$?

Suppose that $A \in \mathbb{R}^{n \times n}$ is symmetric positive definite. In this case, solving $Ax = b$ with $x,b \in \mathbb{R}^{n}$ is equivalent to find \begin{align} \underset{x \in \mathbb{R}^...
In the book I am currently reading, the steepest descent is described as follows: $$\min_{\mathbf{x}} \frac{1}{2}x'Qx - x'b$$ Let this quadratic problem be the initial position and Q must be positive ...