3
$\begingroup$

If the gradient only results in a vector telling you the steepest direction to travel, how can the "slope" in any direction be calculated?

If the gradient is:

$\nabla f(x,y,z) = \left[\begin{array}{c}\frac{\partial f}{\partial x}\\\frac{\partial f}{\partial y}\\\frac{\partial f}{\partial z} \end{array}\right]$

How can the directional derivative simply be a dot product between the gradient and the vector?

$\nabla_{\vec{v}}\,f(x,y,z) = \nabla f(x,y,z) \cdot \vec{v} = \vec{v}_x\frac{\partial f}{\partial x}+\vec{v}_y\frac{\partial f}{\partial y}+\vec{v}_z\frac{\partial f}{\partial z}$

$\endgroup$
  • 1
    $\begingroup$ It isn't just the direction of steepest ascent. The logic goes the other way: the directional derivative is the dot product of the gradient with a vector, and as a consequence the normalized gradient is the direction of steepest ascent. This follows from the Cauchy-Schwarz inequality. $\endgroup$ – Ian Jan 13 at 20:08
2
$\begingroup$

I don’t know how well you already know stuff so I’m going to just justify everything I can. There’s a paragraph about the gradient at the bottom. Also the logic extends so I’ll just talk about functions with 2 inputs (surfaces) instead of three. It might be more helpful. You also might need some visuals and I think there are several things online or I believe khan academy has some helpful visuals even though I’m not always a fan of their regular videos.

Consider the multivariable chain rule. Recall that it works intuitively because the change in z arises from the combined changes in x and in y. Also you can try some practical thought experiments help make it clear, all with the basic idea that for $z(x(t),y(t))$, the change of the height z as you go through time (or more generally some other parameter) is the amount it changes due to x plus the amount it’s changing due to y since they are independent. You could readily substitute amount and rate in that previous sentence, which yields the multivariable chain rule, as was desired.

Now, this $f(x(t),y(t))$ can be interpreted as a path above a graph on the xy plane, which goes along the surface $f$. When the graph (x,y) is a straight line and you move along it with constant speed, you have the derivative as you move in that direction (not just x or y, which are the partial derivatives). If you move at different speeds, like twice the speed, the height changes twice as much in the same amount of time so the directional derivative also doubles. Directional derivatives, however, we take when moving at a constant speed (so they’re like a regular partial derivative but moving in a different direction), thus the vector is normalized. That’s the interpretation of not normalizing the vector (which is just moving at a different speed) cause I don’t feel like normalizing this: suppose $\frac{dy}{dt}=2$, and $\frac{dx}{dt}=3$ (I’ll try to look up and make the partial derivative symbol in there later). As you move along the vector 2,3 in a unit of time, the rate of change of z is these substituted into the multivariable chain rule, which is algebraically equivelant to the gradient dot the vector you move along in a period.

Additionally, the dot product is saying how much length of one vector there is per length of the other along it’s direction, because this is equivalent to projecting one vector onto another and multiplying their lengths. Note: it doesn’t matter which is projected onto which because the dot product is symmetric. Anyways, locally the gradient is the direction of steepest ascent, and dotting a vector with it is just saying how much is this unit motion lined up along this direction of main (positive) ascent, the quantity you’re measuring. This comparison to the gradient is logically the same as saying how much of the increase (gradient) is in a particular direction, hence, directional derivative.

*The gradient is how a function ascends, ie a certain amount in one direction plus a certain amount in the other(s), or: $i\frac{df}{dx}+j\frac{df}{dy}$ (again, sorry about the latex, and that’s meant to be i hat and j hat). Viewed in this light, it only makes sense that “how it’s ascending” ascends in the direction of steepest ascent. It’s just as a gradient of a single variable function is a vector telling you loosely “how it’s changing”, and if you can connect this to the idea that the vector (which is just a scalar in this case) points in the direction of increase according to how it’s increasing you can see the analogy for higher dimensional gradients. Of course you could also prove this analytically in several ways, one of which is working backwards and optimizing the angle of the directional derivative (which weirdly isn’t circular reasoning and doesn’t require you to already know about the gradient).

I haven’t thought through this too much but since every analytic function is has a tangent plane you could figure it out by doing it on a plane.

And again, choosing some real world or more abstract variables with this and doing a thought experiment is fun. And like Mohammad said, like normal slope, it’s a scalar.

$\endgroup$
  • $\begingroup$ Oh I get it. Of course it is akin to interpolation. The tangents in all the directions are on a flat plane and that is why you can use a dot product to describe the directional derivative. Am I right? If so, I can’t believe I did not get this earlier. Thank you. Your second to last paragraph if the one that made it super obvious. $\endgroup$ – Arvin Singh Jan 13 at 23:45
  • $\begingroup$ @ArvinSingh I think so but I have to learn about interpolation. $\endgroup$ – Benjamin Thoburn Jan 14 at 17:52
  • 1
    $\begingroup$ Linear interpolation as in scaling between 2 values $\endgroup$ – Arvin Singh Jan 14 at 20:37
1
$\begingroup$

The result of dot product of gradient vector and the unit direction vector is a scalar not a vector. The scalar is the directional derivative in that direction. I noticed that you have a vector for the result of the dot product instead of a scalar. You need to add the components to get the result.

$\endgroup$
  • $\begingroup$ Sorry, I didn't realize my mistake, but I just don't see how combinations of 3 derivatives can give you derivatives in every direction. Derivatives can't just be linear interpolations, can they? $\endgroup$ – Arvin Singh Jan 13 at 22:04
  • $\begingroup$ Derivatives are slopes which are the instantaneous rate of change in the given direction. $\endgroup$ – Mohammad Riazi-Kermani Jan 13 at 22:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.