Suppose we have a multivariable function $f$ and I want to find it's directional derivative along the direction $\vec u$. Then the formula tells us that the directional derivative will be $$\nabla f\cdot \vec u = |\nabla f||u|cosθ$$ where $θ$ is the angle between these two vectors. That means that, for the directional derivative to be $0$, then $cosθ = 0 \to θ= π/2$.

Is there an intuitive way to understand this? Why the gradient needs to be perpendicular to my chosen direction in order for the directional derivative, i.e.the slope of the function in the particual point along my chosen direction, to be $0$ ?

Thanks in advance.

EDIT : I can maybe understand something related to the contours. https://en.wikipedia.org/wiki/Directional_derivative The first picture here shows a contour plot, where we see that the gradient is perpendicular to the contours and we also see a dir. derivative. If these 2 vectors were perpendicular,then the dir. derivative would have to be tangent to the contour and therefore, our unit vector $u$ would be tangent to it. That means that our direction is tangent to the contour. So for small steps, the function wouldn't change value. So our rate of change would be zero, i.e. the dir. derivative would be zero. Is that correct?

  • 1
    $\begingroup$ The ideas in your edit seem correct. This is one of those cases where since you found an answer on your own that works for you, you could post it as an answer to your own question and even accept it (though if you leave the question with no accepted answer for a while it's a bit more likely that someone else will offer something). $\endgroup$ – David K Nov 5 '19 at 17:39
  • 1
    $\begingroup$ Yeah while I have figured something out, I still dont feel like i totally understand the whole idea. Your answer gave me some insights but I cant really visualize what you saying so I think i'll leave it unanswered for now in hopes that someone else might say something that sparks my understanding. In any case, thanks a lot for your time and effort :)) $\endgroup$ – thenac Nov 5 '19 at 17:42
  • $\begingroup$ That sounds like a good plan. $\endgroup$ – David K Nov 5 '19 at 17:43

In order for the gradient $\nabla f$ to exist at a given point, $f$ has to be differentiable at that point.

For a two-variable function $f,$ this implies that you can graph $f$ in three dimensions ($x$ and $y$ for the input variables, $z$ for the output value) and then there will be a plane tangent to $f$ at the point where you found the gradient. If the plane is horizontal the gradient is zero; otherwise the plane is tilted by rotating around some horizontal line. The fastest way to climb on the plane is to go upward perpendicular to that horizontal line; the gradient is simply the projection of that direction back down onto the $x,y$ plane.

Since the gradient is perpendicular to the horizontal line, the horizontal line is perpendicular to the gradient. So your angle $\theta = \pi/2$ is simply selecting a direction along the horizontal line.

The situation for $f$ with more than two variables is analogous, though harder to visualize graphically.

Some related questions:

Why tangent surface is a plane

Trouble with gradient intuition

Why is the gradient the vector of strongest slope?

  • $\begingroup$ Thanks for the answer and for the links. I'm not sure I can visualize what you're saying. I've edited the post to add something related to this, because I cant type such a long comment. Id be grateful if you could take a look, thanks! $\endgroup$ – thenac Nov 5 '19 at 15:26
  • 1
    $\begingroup$ You seem to have a good visualization using the contour plot. If that works for you then that's probably good enough. Indeed the gradient is perpendicular to the contour lines (I think that's in one of the links) and if you go along a contour line then the function value doesn't change. Since the direction perpendicular to the gradient is (at that instant) the same as the direction along the contour, you get zero rate of change (at that instant) if you go in that direction. $\endgroup$ – David K Nov 5 '19 at 17:35
  • $\begingroup$ Yeah exactly, we always talk about very very small changes. So if I move along the contour line for a very small step, the value of $f$ does not change and I'm still perpendicual to the gradient. It helps me understand a bit better yeah. $\endgroup$ – thenac Nov 5 '19 at 17:38

Actually, I think the "True" issue here is understanding why the dot product formula $(x_1,\dots,x_n) \cdot (y_1, \dots,y_n)=\sum x_iy_i$ is projection for unit vectors. Once you accept this, the gradient condition is quite natural. I recommend this video for understanding this bit :).

Let $f: \mathbb R^n \to \mathbb R$ be a smooth function with gradient $\nabla f$. Then, at a point $\nabla f(p)$ is actually a linear map $T_p(\mathbb R^n) \to T_{f(p)}(\mathbb R)$, which is really just $\mathbb R^n \to \mathbb R$. In other words, it is the first order linear approximation to your function. You can think of $T_p(M)$ as having basis $\partial/\partial x_i$, which is just the tangent vector associated to each direction. In particular, we can write down a matrix for this linear transformation, which is a row vector $[\partial f/\partial x_1, \dots,\partial f/\partial x_n]$. Now, a vector $u$ at that point can be expressed as $u=(u_1, \dots, u_n)$.

In this case, $\nabla f(u)=\begin{pmatrix}\partial f/\partial x_1& \dots&\partial f/\partial x_n \end{pmatrix}\begin{pmatrix}u_1\\ \vdots\\ u_n \end{pmatrix}=\sum \partial f/\partial x_i \cdot u_i=(\nabla f)^T \cdot u$.

In other words, the equation you wrote down follows immediately from the fact that $u \cdot v/ (\|u\|\|v\|)=\cos \theta$.

It follows immediately that $\nabla f(u)=0$ if and only if $(\nabla f)^T \cdot u=0$ iif and only if $(\nabla f)^T \perp u$.

  • $\begingroup$ Thank you for the answer. The video was great, but still I don't really get the intuition of what you're saying. Thank you in any case :) $\endgroup$ – thenac Nov 6 '19 at 19:20
  • 1
    $\begingroup$ Well, like I said if you feel in your bones why the dot product is projection, it really clears up why this should be true. I guess the mental stress is in duality $\endgroup$ – Andres Mejia Nov 6 '19 at 19:25
  • 1
    $\begingroup$ Sorry I couldn’t be more helpful. Please let me know if there is anything that I can clear up for you $\endgroup$ – Andres Mejia Nov 6 '19 at 20:43
  • $\begingroup$ No worries, it's totally fine. :) Regarding the projection, it helped me a bit. It's just that I need to completely grasp the intuitive side, you know. What I wrote in my edit helped me a bit and I think I'm ok for now. I'm sure if I keep thinking on it I'll eventually get it fully haha $\endgroup$ – thenac Nov 7 '19 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.