# Intuition behind gradient VS curvature

In Newton's method, one computes the gradient of a cost function, (the 'slope') as well as its hessian matrix, (ie, second derivative of the cost function, or 'curvature'). I understand the intuition, that the less 'curved' the cost landscape is at some specific weight, the bigger the step we want to take on the landscape terrain. (This is why it is somewhat superior to simply gradient ascent/descent).

Here is the equation:

$$\mathbf {w_{n+1} = w_n - \alpha\frac{\delta J(w)}{\delta w}\begin{bmatrix}\frac{\delta^2J(w)}{\delta w^2}\end{bmatrix}^{-1}}$$

What I am having a hard time visualizing, is what is the intuition behind 'curvature'? I get that is the 'curviness of a function', but isnt that the slope? Obviously it is not, that is what the gradient measures, so what is the intuition behind curvature?

Thanks!

-

Intuitively, curvature is how fast the slope is changing: a greater rate of change of the slope means it is more curved.

So it is related to the derivative of the slope, i.e. the derivative of the derivative or the second derivative.

-
Thanks Henry. I get that - I dont know why I am having a hard time visualizing it. Maybe I need more coffee. When I visually inspect a curve, I am seeing its - gradient, no? Perhaps there is a mis-match between the colloquial usage of curvature in English and curvature in the math? Curvature_english = gradient_math? –  Mohammad Sep 1 '12 at 14:15
@Mohammad: A function with a fixed slope (constant first derivative, zero second derivative) would be a straight line with no curvature, even though its values change because of the slope. –  Henry Sep 1 '12 at 14:43

The gradient and the radius of curvature at a point x are both normal to the equipotential surface through x.

The radius of curvature is the radius of the largest sphere that is tangent to the surface at that point. It has nothing to do with the slope, but only with the equipotential surface. Curvature is the reciprocal of the radius of curvature. It is used rather than the radius of curvature because the radius of curvature is infinite when curvature is 0. 0 is more convenient to use in computations than infinity.

The gradient is a vector that points "up hill" and has magnitude equal to the slope.

-