Tell me more ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

In Newton's method, one computes the gradient of a cost function, (the 'slope') as well as its hessian matrix, (ie, second derivative of the cost function, or 'curvature'). I understand the intuition, that the less 'curved' the cost landscape is at some specific weight, the bigger the step we want to take on the landscape terrain. (This is why it is somewhat superior to simply gradient ascent/descent).

Here is the equation:

$$ \mathbf {w_{n+1} = w_n - \alpha\frac{\delta J(w)}{\delta w}\begin{bmatrix}\frac{\delta^2J(w)}{\delta w^2}\end{bmatrix}^{-1}} $$

What I am having a hard time visualizing, is what is the intuition behind 'curvature'? I get that is the 'curviness of a function', but isnt that the slope? Obviously it is not, that is what the gradient measures, so what is the intuition behind curvature?

Thanks!

share|improve this question

1 Answer

up vote 1 down vote accepted

Intuitively, curvature is how fast the slope is changing: a greater rate of change of the slope means it is more curved.

So it is related to the derivative of the slope, i.e. the derivative of the derivative or the second derivative.

share|improve this answer
Thanks Henry. I get that - I dont know why I am having a hard time visualizing it. Maybe I need more coffee. When I visually inspect a curve, I am seeing its - gradient, no? Perhaps there is a mis-match between the colloquial usage of curvature in English and curvature in the math? Curvature_english = gradient_math? – Mohammad Sep 1 '12 at 14:15
@Mohammad: A function with a fixed slope (constant first derivative, zero second derivative) would be a straight line with no curvature, even though its values change because of the slope. – Henry Sep 1 '12 at 14:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.