I'm having trouble understanding the proof of directional derivative and the gradient. Could someone give me a easy-to-read proof of the directional derivative and explain why does the gradient point to the direction of maximum increase?

Thank you very much for any help! =)

  • $\begingroup$ What do you mean by "proof of the directional derivative"? $\endgroup$ – gerw Apr 19 '13 at 7:09
  • $\begingroup$ Hi, I mean where did the definition / formula for it come from? Why is it the way it is. en.wikipedia.org/wiki/Directional_derivative . I just know the definition...I want to know WHY it is defined the way it is =) $\endgroup$ – jjepsuomi Apr 19 '13 at 7:26
  • $\begingroup$ Did I make myself clear? =) $\endgroup$ – jjepsuomi Apr 19 '13 at 7:45

As for the gradient pointing in the direction of maximum increase, recall that the directional derivative is given by the dot product

$$\nabla f(x)\cdot\textbf{u},$$

where $$\nabla f(x)$$ is the gradient at the point $\textbf{x}$ and $\textbf{u}$ is the unit vector in the direction we are considering. Recall also that this directional derivative is the rate of increase/decrease of the function in the direction of $\textbf{u}$ at the point $\textbf{x}$. The dot product has two equivalent definitions:

$$\textbf{u}\cdot\textbf{v}=u_1v_1+u_2v_2+...+u_nv_n$$ or $$\textbf{u}\cdot\textbf{v}=||\textbf{u}||||\textbf{v}||cos(\theta),$$ where $\theta$ is the angle between the vectors.

Using this second definition, the fact that $\textbf{u}$ is a unit vector, and knowledge of trigonometry, we see that the directional derivative $D\textbf{u}$ at $\textbf{x}$ is bounded: $$-||\nabla f(x)||=cos(\pi)||\nabla f(x)||\leq cos(\theta)||\nabla f(x)||$$ $$\leq cos(0)||\nabla f(x)||=||\nabla f(x)||$$

Since $0\leq||\nabla f(x)||$, we see that the maximum rate of change must occur when $\theta=0$, that is, in the direction of the gradient.

As for the directional derivative consider the direction of the vector $(2,1)$. We want to know how the value of the function changes as we move in this direction at a point $\textbf{x}$. Well, for infinitesimally small units, we are moving $2$ units in the $x$ direction and $1$ unit in the $y$ direction so the change in the function is $2$ times the change in $f$ that we get as we move $1$ unit in the $x$ direction plus $1$ times the change in $f$ that we get as we move $1$ unit in the $y$ direction. Finally, we divide by the norm of $(2,1)$ so that we have the change for $1$ unit in this direction.

  • $\begingroup$ Thank you for your answer =) I changed the grad to $\nabla$ =) $\endgroup$ – jjepsuomi Apr 19 '13 at 8:01
  • $\begingroup$ You're welcome and thanks for the edit. It looks better with the nabla. $\endgroup$ – jim Apr 19 '13 at 8:13

As for why the gradient points in the direction of maximum increase, let's say we don't know this and we want to find a unit vector $\vec{u}$ such that the directional derivative of some function $f$ is the greatest in the direction of $\vec{u}$. Then if $\theta$ is the angle between $\nabla f$ and $\vec{u}$ we have $\nabla f_{\vec{u}}=\nabla f \cdot \vec{u}=|\nabla f||\vec{u}|\cos(\theta)=|\nabla f|\cos(\theta)$ since $\vec{u}$ is a unit vector. This quantity is then maximized when $\cos(\theta)=1$, i.e., when $\theta=0$, thus $\vec{u}$ points in the direction of the gradient.

  • $\begingroup$ Thank you very much! What about the formula of directional derivative? Why the formula is the way it is? Can you help me see it intuitively? =) $\endgroup$ – jjepsuomi Apr 19 '13 at 7:28
  • 2
    $\begingroup$ Think of the usual partial derivatives as derivatives in the direction of the coordinate axes, and notice that for example $\frac{\partial f}{\partial x}=\nabla f \cdot (1,\ldots,0)$. This should guide your intuition for the general case. $\endgroup$ – dezign Apr 19 '13 at 7:35
  • $\begingroup$ Thank you for your answer! +1 $\endgroup$ – jjepsuomi Apr 19 '13 at 7:46

Although the previous 2 answers are the "real" answers to your inquiry I feel that perhaps you were asking for an intuitive motivation for why grad f represents the maximum increase. The reason lies in the formal reasoning above... Think about what theta being zero really says... that grad f points in the direction orthogonal to (perpendicular) to the contour lines.. higher regions will have contour lines that get closer together and more dense and if grad f if orthogonal to these lines it must be pointing straight up.

  • $\begingroup$ Thank you for your answer! =) +1 $\endgroup$ – jjepsuomi Apr 20 '13 at 7:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.