I imagine it as if one is going up a physical hill. It doesn't seem like there's a guarantee that going in the opposite direction of greatest increase in height will necessarily be the direction of greatest decrease in height.

  • 3
    $\begingroup$ The crucial point here is that a physical hill doesn't have to be differentiable. Note that a differentiable function can be approximated by a hyperplane at the point which you are looking at. That the statement is true for hyperplanes in at least $\mathbb R^3$ should be intuitive again. $\endgroup$ – Tim B. May 20 '15 at 18:37
  • $\begingroup$ There are many similar questions already on stackexchange. Compare Why gradient descent works?. $\endgroup$ – David K May 20 '15 at 18:38
  • 1
    $\begingroup$ Lebtz is right: the key is to think about what "greatest ascent/descent" actually means. How is it measured? In this case, to measure the rate of ascent up a hill in a direction $v$ means to first draw the tangent plane to the hill at your current location, then measure the slope of the plane in the $v$ direction. And now it should be intuitive that the slope in the $-v$ direction is opposite that in the $v$ direction. $\endgroup$ – user7530 May 20 '15 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.