32
$\begingroup$

Consider a function $g: \mathbb{R^3} \to \mathbb{R}$ defined by $g(x,y,z) = z^2 -x^3 + 2z + 3y^3$

Find the gradient of $g$ at the point $(2,1,-1)$.

Is the gradient $\nabla g(2,1,-1)$ given by a vector, that is, $\nabla g(2,1,-1) = -12i + 9j$? If so, then what does the directional derivative mean?

$\endgroup$

7 Answers 7

36
$\begingroup$

The gradient is a vector; it points in the direction of steepest ascent.

The directional derivative is a number; it is the rate of change when your point in $\Bbb R^3$ moves in that direction. (You can imagine "reducing" your function to a function of a single variable, say $t$, by "slicing" the curve in that direction; the directional derivative is just then the 1-D derivative of that "sliced" function.)

$\endgroup$
1
  • $\begingroup$ steepest ascent, does that mean that for example $\nabla f = (a,b)$ gives us the fastest growing direction $(a,b)$. And from this that we should increase $x$ and $y$ with a scalar times $a$ and $b$ respectively in order to increase the function value $f$ as fast as possible? $\endgroup$
    – dekuShrub
    Commented May 22, 2018 at 15:37
19
$\begingroup$

Be careful that directional derivative of a function is a scalar while gradient is a vector.

The only difference between derivative and directional derivative is the definition of those terms. Remember:

  • Directional derivative is the instantaneous rate of change (which is a scalar) of $f(x,y)$ in the direction of the unit vector $u$.
  • Derivative is the rate of change of $f(x,y)$, which can be thought of the slope of the function at a point $(x_0,y_0)$.
$\endgroup$
16
$\begingroup$

Yes, the gradient is given by the row vector whose elements are the partial derivatives of $g$ with respect to $x$, $y$, and $z$, respectively. In your case the gradient at $(x,y,z)$ is hence $[-3x^2,9y^2,2z+2]$. The gradient at $(2,1,-1)$ is therefore $[-12,9,0]$.

The directional derivative at a point $(x,y,z)$ in direction $(u,v,w)$ is the gradient multiplied by the direction divided by its length. So if $u^2+v^2+w^2=1$ then the directional derivative at $(x,y,z)$ in direction $(u,v,w)$ is just $-3x^2 u + 9y^2v + (2z+2)w$.

If $u^2+v^2+w^2\neq 1$ then you should divide the number above by $\sqrt{u^2+v^2+w^2}$.

In sum, the gradient is a vector with the slope of the function along each of the coordinate axes whereas the directional derivative is the slope in an arbitrary specified direction.

$\endgroup$
5
$\begingroup$
  • A Directional Derivative is a value which represents a rate of change
  • A Gradient is an angle/vector which points to the direction of the steepest ascent of a curve.

Let us take a look at the plot of the following function:

$$ \bbox[lightgray] {f(x) = -x^2+4}\qquad (1)$$

enter image description here

The 1st derivative of the function is:

$$ \bbox[lightgray] {\frac{dy}{dx} = -2x}\qquad (2)$$

Putting $x=-1$ in $(2)$ we obtain,

$\implies \frac{dy}{dx} = \frac{rise}{run}= 2 ... ... ...\qquad (3)$

Also,

$tan \theta = 2$
$\implies \theta = tan^{-1}(2)$
$\implies \theta = 0.964 $ radian

So, $\theta = 55.23 ^\circ ... ... ...\qquad (4)$

.

Similarly, putting $x=-2$ in $(2)$ we obtain,

$\implies \frac{dy}{dx} = \frac{rise}{run}= 4 ... ... ...\qquad (5)$

So, $\theta = 57.25 ^\circ ... ... ...\qquad (6)$

enter image description here

So, the answer is:

  • (3) and (5) are Directional Derivatives.
  • Directional Derivatives are scalar values.

And,

  • (4) and (6) are Gradients.
  • Gradients are vector values.
$\endgroup$
4
  • $\begingroup$ Very good illustration. Makes the point clearly. $\endgroup$
    – naive
    Commented Mar 11, 2020 at 8:06
  • 1
    $\begingroup$ This is an old question, yet this incorrect answer still has +6 upvote. The context of the question is multivariable calculus, and this answer does not make sense in that context. $\endgroup$
    – user26672
    Commented Mar 4, 2021 at 4:53
  • $\begingroup$ So in deep learning, when we calculate the gradient, are we actually calculating gradient or derivative? Because we are calculating by how much we should move in a certain direction for optimization, right? $\endgroup$ Commented Jul 13, 2021 at 18:14
  • 1
    $\begingroup$ How does (6) 𝜃=57.25... represent a vector value? Looks like a scalar to me... $\endgroup$
    – Noumenon
    Commented Mar 29, 2023 at 4:12
2
$\begingroup$

Taking a function $g: \mathbb{R}^3\rightarrow \mathbb{R}$. Its gradient $\nabla{g}$ is a vector, while the directional derivative is the inner product of another vector $\mathbf{v}$ in the same space with $\nabla g$, denoted as $\mathbf{v}\cdot\nabla g$. It means the projection of the vector $\nabla g$ onto the direction of the vector $\mathbf{v}$, which represents how the function $g$ varies in that direction.

$\endgroup$
1
$\begingroup$

In simple words, directional derivative can be visualized as slope of the function at the given point along a particular direction. For example partial derivative w.r.t x of a function can also be written as directional derivative of that function along x direction.

Gradient is a vector and for a given direction, directional derivative can be written as projection of gradient along that direction.

Please go through this link to have a clear idea: http://omega.albany.edu:8008/calc3/directional-derivatives-dir/define-m2h.html

$\endgroup$
1
$\begingroup$

To Put it very simply: the gradient is a vector that has both a magnitude and a direction, while the derivative is a scalar that only has a magnitude.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .