Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

In Newton's method, one computes the gradient of a cost function, (the 'slope') as well as its hessian matrix, (ie, second derivative of the cost function, or 'curvature'). I understand the intuition, that the less 'curved' the cost landscape is at some specific weight, the bigger the step we want to take on the landscape terrain. (This is why it is somewhat superior to simply gradient ascent/descent).

Here is the equation:

$$ \mathbf {w_{n+1} = w_n - \alpha\frac{\delta J(w)}{\delta w}\begin{bmatrix}\frac{\delta^2J(w)}{\delta w^2}\end{bmatrix}^{-1}} $$

What I am having a hard time visualizing, is what is the intuition behind 'curvature'? I get that is the 'curviness of a function', but isnt that the slope? Obviously it is not, that is what the gradient measures, so what is the intuition behind curvature?

Thanks!

share|improve this question
add comment

2 Answers 2

up vote 1 down vote accepted

Intuitively, curvature is how fast the slope is changing: a greater rate of change of the slope means it is more curved.

So it is related to the derivative of the slope, i.e. the derivative of the derivative or the second derivative.

share|improve this answer
    
Thanks Henry. I get that - I dont know why I am having a hard time visualizing it. Maybe I need more coffee. When I visually inspect a curve, I am seeing its - gradient, no? Perhaps there is a mis-match between the colloquial usage of curvature in English and curvature in the math? Curvature_english = gradient_math? –  Mohammad Sep 1 '12 at 14:15
    
@Mohammad: A function with a fixed slope (constant first derivative, zero second derivative) would be a straight line with no curvature, even though its values change because of the slope. –  Henry Sep 1 '12 at 14:43
add comment

The gradient and the radius of curvature at a point x are both normal to the equipotential surface through x.

The radius of curvature is the radius of the largest sphere that is tangent to the surface at that point. It has nothing to do with the slope, but only with the equipotential surface. Curvature is the reciprocal of the radius of curvature. It is used rather than the radius of curvature because the radius of curvature is infinite when curvature is 0. 0 is more convenient to use in computations than infinity.

The gradient is a vector that points "up hill" and has magnitude equal to the slope.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.