I've read that the directional derivative is the rate of change of a function $f$ in a given direction $\mathbf{v}$, given as $\nabla f\cdot \mathbf{v}$. I've also read (perhaps incorrectly) that the magnitude of the gradient also tells us the rate of change. If so, what does the directional derivative of the gradient, i.e. $\nabla f\cdot \nabla f$ tell us?

The magnitude of the gradient is the maximum rate of change at the point. The directional derivative is the rate of change in a certain direction. Think about hiking, the gradient points directly up the steepest part of the slope while the directional derivative gives the slope in the direction that you choose to walk.

There's more than one direction starting at a point (you're in a multivariate situation). Therefore, it doesn't make sense to talk about "the rate of change." Each direction of travel gives a different rate of change. The magnitude of the gradient is the largest of these rates of change while the directional derivative is the rate of change in a particular direction.

Instead of $\nabla f\cdot \nabla f$, you might be interested in the following. Let $u$ be a unit vector which points in the direction of $\nabla f$. Then the directional derivative in the direction of $u$ is $\|\nabla f\|$, which is the maximum possible rate of change.

• I understand, but I'm having difficulty making sense of the directional derivative in the direction of the gradient, i.e. $\nabla f\cdot\nabla f$. How do we interpret this vs. the magnitude of the gradient? Which one is the actual rate of change? – John Dec 22 '16 at 15:20
• The directional derivative in the direction of the gradient will give you $|\nabla f |^2$. To talk about rate of change (in the direction of some vector $\textbf{v}$), you could divide the directional derivative in the direction of $\textbf{v}$ by the length of $\textbf{v}$ - that would give you the slope of graph in that particular direction. So the slope of the graph of $f$ in the direction of $\nabla f$ would then be simply $|\nabla f|$. – john Dec 22 '16 at 15:36
• The directional derivative in the direction of the gradient is the rate of change when you travel in the direction of the gradient with speed $\|\nabla f\|$. – Michael Burr Dec 22 '16 at 15:43
• If I understand correctly, when you say greatest rate of change, you mean at an infinitesimally local scale. If that's the case, even if we moved a unit vector along the direction of the gradient we can't guarantee that we'll see a change in $f$ of size $||\nabla f||$. And even more so if we moved in the direction of gradient with a vector $\mathbf{v}$ such that $||\mathbf{v}||=||\nabla f||$, right? – John Dec 22 '16 at 17:58

Imagine you are standing on the side of a hill (the hill is the graph of your function $f$) with latitude and longitude $(x, y)$. The gradient vector $\nabla f(x, y)$ tells you the direction in which the hill is steepest. In other words, if you took a step in the direction of the gradient, you would be going "straight up" the hill.

A slightly different question might be "If I look in some direction, say north east, how steep is the hill if took a step in that direction?" That is the directional derivative in the "north east" direction.

Putting these things together, we can compute the $\textit{maximum possible steepness}$ of the hill by computing the directional derivative in the steepest possible direction, namely in the direction of the gradient. The way the numbers work out, this maximum possible steepness is equal to the magnitude of the gradient at that point.

So, in more mathematical terms, $\textit{maximum}$ rate of change of a function $f$ at a point $(x, y)$ is equal to $\|\nabla f(x, y)\|$.

• Does that mean the magnitude of the gradient cannot be negative? – John Dec 22 '16 at 18:12
• @John Yes, the magnitude of any vector cannot be negative. Remember that magnitude means length. Negative lengths don't really make sense. – ChocolateAndCheese Dec 22 '16 at 18:22
• I think I phrased my question rather poorly. I'm having a hard time intuitively grasping how moving in the direction of the gradient will never give a decrease in $f$. It makes sense from the definition of the dot product and directional derivatives, but on a component level, many or perhaps all partial derivatives can be negative, so how does moving in their direction given an increase? – John Dec 24 '16 at 12:50
• @John If all partial derivatives at a point are negative, then you can move in the direction of a vector with all negative components, which would yield a positive directional derivative. For a concrete example, consider $f(x) = -x^2 - y^2$ at the point $(1, 1)$. – ChocolateAndCheese Jan 3 '17 at 17:07
• @ChocolateAndCheese Why the directional derivative changes sign if we move in opposite direction? It doesn't make sense. I mean even in single variable calculus derivative doesn't change sign if we move in the negative axis. – user599310 Nov 15 '20 at 0:11