Why is the gradient the vector of strongest slope? [duplicate]

Question

I'm struggling with the interpretation of the gradient. The gradient of a function is orthogonal to the tangent on a particular point, which makes perfect sense from its definition (its scalar product with the derivative = 0)

But I have also read many time that the gradient can be seen as the vector with the strongest slope. I have drawn a sketch, to try to understand this, with an univariate function (of variable x). MM'=dx.

Here, you can see that it has nothing to do with the slope of the function, since it's not following it, it's orthogonal. What did I misunderstood in this concept?

Answer 1

For your own sake, as soon as possible please disabuse yourself of this extremely misguided notion you have about what the gradient is orthogonal to. I picked up a similar notion in my freshman year of college that turned out to be a huge stumbling block to my progress in calculus. Among all misconceptions that ever hindered me in mathematics, this one sticks out in my memory as the most harmful.

You can indeed define a gradient of a single-variable function, but the gradient at any point $x$ is a single real number, not a two-dimensional vector as you have attempted to depict it in your graph. If you must indicate a "direction" of the gradient in this example, it should be parallel to the $x$ axis.

The gradient is not orthogonal to any interesting property of the function unless the function has at least two input variables. And then the thing you want to look for are the function's level curves (for a two-variable function) or its level surfaces (for more variables).

An analogy that might help you start thinking about these things is a contour map, on which the contour line for $100$ meters (for example) goes through all the points whose altitude is $100$ meters. If you built a bicycle trail along a contour line it would be nice and level, not requiring you to pedal uphill or use the brakes to avoid going downhill too fast. Now if the map has a grid showing you the meters east or north of some reference point, you can read $x$ and $y$ coordinates from the grid and let altitude be a function of these, $f(x,y).$ Then the contour lines of the map are the level curves of $f$ and the gradient of $f$ at any point on a contour line can be represented by projecting the contour line down onto your theoretical $x,y$ plane and taking a vector orthogonal to the image of the contour line in that plane.

If you add a suitable $z$-component to this gradient vector, you can get a new vector that actually is orthogonal to the surface of the hill at the $x,y$ coordinates where you found the gradient. The "orthogonal" vector on your graph is that kind of orthogonal vector. Just remember that this new vector is not the gradient.

I like to think of the gradient of a function in the following way: the gradient lies completely in the space of the input to the function (so for a function of $x$ and $y,$ the only components of the gradient are an $x$-component and a $y$-component), the direction of the gradient is the direction to go within the input space in order to get to greater values of the function as quickly as possible, and the magnitude of the gradient represents how quickly the value of the function grows if you go in that direction.

Answer 2

The gradient is a generalization of the usual concept of derivative to functions of several variables. The use of the word ''gradient'' for a function of a single variable in an abuse that can be justified beacuse this term can indicate also the slope of a stright line.

If properly defined for multi-variable functions, than the gradient is really the vector that gives the direction of strongest slope.

Answer 3

The gradient is useful to calculate the directional derivative of a scalar field. If you denote the directional derivative in $\vec a$ along an unitary vector $\vec{y}$ as $f'(\vec a;\vec y)$, you have

$$f'(\vec a;\vec y) = \nabla f(\vec a)\cdot \vec y = ||\nabla f(\vec a)|| \space ||\vec y ||\cos\theta = ||\nabla f(\vec a)|| \cos\theta $$

In which $\theta$ is the angle between the gradient and $\vec y$. Therefore, the directional derivative will be maximum when $\theta$ is $0$ or $\pi$. That is, the maximum change in $f$ is in the same direction of the gradient. Look at the case in $\mathbb{R}^2$: