# Newton's Method vs Gradient Descent?

Currently I am studying logistic regression.

I read online that Gradient Descent is a 1st-order optimisation algorithm, and that Newton's Method is a 2nd order optimisation algorithm.

Does that mean that Gradient Descent cannot be used for multivariate optimisation and that Newton's Method cannot be used for univariate optimisation? Or can Newton's Method be done using a 1st order Taylor polynomial and still be different from Gradient Descent?

These sites are causing me to question:

• You can certainly use both gradient descent and Newton's method for either univariate or multivariate optimization. – littleO Nov 27 '19 at 11:36
• This means that gradient descent depends only on the derivative of a function, whereas Newton's method depends on the first and the second derivative. This terminology (1st order/ 2nd order) has nothing to do with univariate or multivariate optimisation. – Slup Nov 27 '19 at 11:37

• @HughPearse Yes. But to see why you have to consider the Newton-Raphson method to solve a system of nonlinear equations in several variables, using the derivative (i.e. the jacobian matrix). When you apply this method to an optimiztion problem, you are given a function $f$ of several variables, and you look for critical points, so the system do solve is $\frac{\partial f}{\partial x_i}=0$, $i=1\dots n$. Thus when you apply Newton's method to this system of equations, you need the second derivative. – Jean-Claude Arbaut Nov 27 '19 at 12:55