I understand that using a unit vector (of a vector say $\vec{a}$ ) and computing the directional derivative gives the slope (or rate of change of the function) in the direction of the vector.

I have three questions :

  1. If I use the vector itself rather than it's unit vector what will I get when I compute it's dot product with the gradient of the function? It wouldn't give the slope of the curve(formed by the slicing of the function with the plane containing the vector $\vec{a}$ ), would it?

Note: The function is scalar.

Also going by it's formal definition:

$\displaystyle \nabla _{\mathbf {v}}{f}({\mathbf {x}})=\lim _{h\rightarrow 0}{\frac {f({\mathbf {x}}+h{\mathbf {v}})-f({\mathbf {x}})}{h}}$

$\mathbf {v}$ is a vector

Quoting from Wikipedia

This definition is valid in a broad range of contexts, for example where the norm of a vector (and hence a unit vector) is undefined.

  1. What does that mean?

Also quoting from Wikipedia:

If the function f is differentiable at x, then the directional derivative exists along any vector v, and one has

$\displaystyle \nabla _{\mathbf {v} }{f}({\mathbf {x} })=\nabla f({\mathbf {x} })\cdot {\mathbf {v} }$

Intuitively, the directional derivative of f at a point x represents the rate of change of f, in the direction of v with respect to time, when moving past x.

  1. Why is it mentioned with respect to time isn't it with respect to the change in x (or/and y ) in the direction of the vector ?

If you define $\nabla_x f(x_0)=\lim_{h \to 0^+} \frac{f(x_0+hx)-f(x_0)}{h}$, then you have the identity $\nabla_x f(x_0)=\| x \| \nabla_{x/\| x \|} f(x_0)$. (I will remark that this notation clashes with notation elsewhere in math, but I will stick with it here.) That is, the derivative "along $x$" is the directional derivative multiplied by the norm of $x$. In effect instead of just moving in a direction and measuring the change in $f$ relative to the distance you traveled in that direction, you are moving in a direction at a particular rate in time and measuring the change in $f$ relative to that change in time. The speed is the conversion factor between these measurements.

This definition of $\nabla_x$ doesn't depend on there being such a thing as the norm of $x$, whereas the directional derivative does. But for your purposes you can ignore this remark for now.

I said this in the first paragraph, but just to directly address your third question, let me add one more thing. The directional derivative does not really have a notion of time, it is really a change in $f$ with respect to distance traveled in the specified direction. Your generalized notion $\nabla_x$ effectively involves time after you identify $\| x \|$ as a speed and $h$ as a time, so that $hx$ is a displacement and $h \| x \|$ is a length.

  • $\begingroup$ Thanks. Can you help me with my first question? $\endgroup$ – paulplusx Aug 2 '18 at 11:44
  • $\begingroup$ @paulplusx Read the first sentence. $\endgroup$ – Ian Aug 2 '18 at 12:49
  • $\begingroup$ I am sorry but I am not able to understand (with the given explanation) what is the exact output of the dot product of gradient and the vector. Could you provide a more intuitive/geometrical answer supported by a bit of maths? (As simple as possible ) $\endgroup$ – paulplusx Aug 2 '18 at 12:59
  • $\begingroup$ @paulplusx It is the directional derivative in the direction of $x$, multiplied by $\| x \|$. The point is that you go $\| x\|$ times further from the start point than you would to estimate the directional derivative for a given $h$, but then you divide by the same $h$, so the ratio is about $\| x\|$ times bigger (which becomes exact for $h \to 0$). $\endgroup$ – Ian Aug 2 '18 at 13:04
  • 1
    $\begingroup$ @john Yes, I was lazy about the notation. $\endgroup$ – Ian Aug 11 '20 at 17:19

"This definition is valid in a broad range of contexts, for example where the norm of a vector (and hence a unit vector) is undefined.

What does that mean?"

Not all vector spaces have a defined inner product space or a norm for that space. But, returning to first principles we can define a directional derivative.

"Intuitively, the directional derivative of f at a point x represents the rate of change of f, in the direction of v with respect to time, when moving past x.

Why is it mentioned with respect to time isn't it with respect to the change in x (or/and y ) in the direction of the vector ?"

If we are looking at the changes in $f(\bf{x})$ as $\bf x$ traverses some path, we may find that we parameterize that path, and might like to think of that parameter as "time."

  • $\begingroup$ Understood. Can you help me with my first question (which is the subject of the question) about non-unit vector dot product with the gradient? $\endgroup$ – paulplusx Aug 2 '18 at 11:45

Its always better to understand vectors and their derivatives in some context taken from physics.

  • directional derivative of distance w.r.t time gives you velocity in the respective direction (like x or y axis/direction). Its a differentiation w.r.t to time. Also, the vector remains a vector after this operation (both distance and velocity have components on the axes in space).

  • gradient of voltage (where we differentiate w.r.t distance) gives you electric field in a particular direction. Here, the operation converts a scalar to a vector. Though voltage is dependent on the position in space, it has a value but no direction. (Just like its hotter when closer to a furnace, there's higher voltage when closer to a positive charge).

So, the two derivatives

  • are not the same

  • are used for different reasons

  • best understood in a given context (because most often math is a means to an end, and to what end?... right?).

Answers to your questions :

1) dot product of (vector, gradient of the function) ... please note that you can't compute gradient of a vector.

2) the 'h' mentioned must be infinitesimal time being multiplied with 'v' (velocity in the x direction). Hence, it is indeed differentiation w.r.t to time.

3) This is answered in 2.

  • $\begingroup$ I meant a scalar function. I have added a note for it now. $\endgroup$ – paulplusx Aug 2 '18 at 17:00

The following definition should explain the directional derivative of a non-unit vector; we can call it the general directional derivative. It is similar to the 'unit vector' definition of the directional derivative given in most intro-calculus textbooks, but scaled by the magnitude of the vector. $$\displaystyle \nabla _{\mathbf {v} }{f}({\mathbf {x} })= \nabla f({\mathbf {x} })\cdot {\mathbf {v} } = \nabla f({\mathbf {x} })\cdot \left( {\frac{\mathbf {v}}{\| \mathbf{v} \| } \|\mathbf{ v}\| }\right) = \|\mathbf{ v}\| ~\nabla f({\mathbf {x} })\cdot \left( {\frac{\mathbf {v}}{\| \mathbf{v} \| } } \right)$$ In the case that $\mathbf {v}$ is already a unit vector, this simplifies to $\nabla f({\mathbf {x} })\cdot {\mathbf {v} }$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.