In my multivariable math class we have been assigned a question which asks:

In what direction does W increase most rapidly at P? What is the value of the maximal directional derivative at P? (where W is a continuous, differentiable function and P is a point)

I understand how to find the direction at which W increases most rapidly (the Gradient Vector at P), and how to interpret this, but am confused as to why the value of the maximal directional derivative at P holds any significance - why is the magnitude of a vector which is only useful for its direction of any importance?

Thanks, JM

  • $\begingroup$ Consider the function $f(x,y)=x$. In the direction of $y$, it doesn't increase, while it does in the direction of $x$. $\endgroup$ – Wojowu Mar 19 '17 at 20:57
  • 1
    $\begingroup$ In what sense is it only useful for its direction? $\endgroup$ – spaceisdarkgreen Mar 19 '17 at 21:00
  • $\begingroup$ The value of a directional derivative isn’t a vector, it’s a scalar, namely, the rate of change of the function in a specific direction. $\endgroup$ – amd Mar 20 '17 at 1:07
  • $\begingroup$ Hmm... are you perhaps confusing the vector that specifies the direction in which the derivative is being calculated with the value of that derivative? $\endgroup$ – amd Mar 20 '17 at 2:45
  • $\begingroup$ I should’ve started with: What’s your definition of directional derivative? $\endgroup$ – amd Mar 20 '17 at 2:59

The value of the directional derivative in its direction of greatest increase (i.e., as you said, along the direction of the gradient) is just that. It is the rate of change of the function along that direction. This answers the question, 'if I move a tiny bit $\epsilon$ in the direction of greatest increase, how does does the function change?' The answer is it changes by (epsilon)*(the derivative in the direction of greatest increase).

It seems something is confusing you so perhaps a quick recap of the directional derivative is in order. Say to avoid clutter we're considering the derivative at the origin. Then we have the first order Taylor expansion $$ f(\mathbf x) \approx f(\mathbf 0) + \nabla f(\mathbf 0)\cdot\mathbf x$$ where $\nabla f$ is the gradient. Recall that the directional derivative of $f$ in the direction of a unit vector $\hat{\mathbf u}$ is $D_{\hat{\mathbf u}}f = \nabla f\cdot\hat{\mathbf u}.$ Thus if we move a small amount $\epsilon $ in the direction $\hat{\mathbf u}$ from the origin then $\mathbf x = \epsilon\hat{\mathbf u}$ and we have $$ f(\mathbf x)-f(\mathbf 0) = \nabla f\cdot(\epsilon\hat{\mathbf u}) = \epsilon D_{\hat{\mathbf u}}f(0).$$

Why is the gradient the direction of greatest derivative? Well, the derivative is the dot product of the gradient with the direction... of course that's largest when the direction is parallel to the gradient.

And what is the value of the derivative at in this direction? It's just the gradient dotted into the unit vector in the same direction, so it's the magnitude of the gradient. Going a little more explicitly, the direction of maximum increase is $\hat{\mathbf u}_{max} = \frac{\nabla f}{|\nabla f|}$ so we have a maximal derivative $$D_{max} = \nabla f\cdot \hat{\mathbf u}_{max} = \frac{\nabla f\cdot \nabla f}{|\nabla f|} = |\nabla f|. $$ So the magnitude of the gradient represents the rate of change in the direction of greatest increase. And as discussed before, the direction of the gradient is the direction of greatest increase. Thus, both the magnitude and direction have nice interpretations.

  • $\begingroup$ Essentially, you say that the gradient is the direction of greatest (directional) derivative because we’ve happened to define directional derivatives so that this is true. Seems a bit circular to me and presupposes the fact that the gradient points in the direction of maximal increase. There’s a more basic definition of directional derivative that doesn’t depend on having defined the gradient first. $\endgroup$ – amd Mar 20 '17 at 0:12
  • $\begingroup$ @amd It's not circular at all. Sure I effectively define the directional derivative in terms of the gradient, but I show on the next line that that definition corresponds to the directional derivative behaving like it should. One could equally view that line as the definition. It doesn't presuppose the gradient being the direction of maximum increase at all... if you view that as the motivation for the definition, sure, but my motivation is that plugging $\epsilon \hat u$ into the taylor series should give the directional derivative times $\epsilon.$ $\endgroup$ – spaceisdarkgreen Mar 20 '17 at 0:37
  • $\begingroup$ Seems like there’s something missing either way. Starting from $\nabla f\cdot\mathbf u$, it’s a trivial consequence of the definition that this is maximized when $\mathbf u$ is parallel to $\nabla f$, but the claim that this is also the max growth rate of $f$ requires that $\nabla f$ have the right properties (which, I guess can be assumed based on what the OP wrote). Starting from $f(\mathbf v+\epsilon\mathbf u)-f(\mathbf v)=\epsilon D_{\mathbf u}f+o(\epsilon)$ or something similar, there’s still a missing connection to be made to $\nabla f$. $\endgroup$ – amd Mar 20 '17 at 2:42
  • $\begingroup$ You yourself write: “Why is the gradient the direction of greatest derivative? Well, the derivative is the dot product of the gradient with the direction.” You don’t think this is even a bit circular, or at least tautological? $\endgroup$ – amd Mar 20 '17 at 2:43
  • $\begingroup$ @amd The connection to the gradient is in my starting point, that the gradient is the vector of coefficients of the first order taylor series / tangent plane. I did not show that its components are the partial derivatives. This makes sense but merits proof. Perhaps this is where your perceived gap is? Everything else, including the relationship to the directional derivative (which is 'obviously' $\nabla f \cdot u$ from this angle), follows. $\endgroup$ – spaceisdarkgreen Mar 20 '17 at 4:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.