Understanding directional derivative and the gradient

I'm having trouble understanding the proof of directional derivative and the gradient. Could someone give me a easy-to-read proof of the directional derivative and explain why does the gradient point to the direction of maximum increase?

Thank you very much for any help! =)

• What do you mean by "proof of the directional derivative"? – gerw Apr 19 '13 at 7:09
• Hi, I mean where did the definition / formula for it come from? Why is it the way it is. en.wikipedia.org/wiki/Directional_derivative . I just know the definition...I want to know WHY it is defined the way it is =) – jjepsuomi Apr 19 '13 at 7:26
• Did I make myself clear? =) – jjepsuomi Apr 19 '13 at 7:45

As for the gradient pointing in the direction of maximum increase, recall that the directional derivative is given by the dot product

$$\nabla f(x)\cdot\textbf{u},$$

where $$\nabla f(x)$$ is the gradient at the point $\textbf{x}$ and $\textbf{u}$ is the unit vector in the direction we are considering. Recall also that this directional derivative is the rate of increase/decrease of the function in the direction of $\textbf{u}$ at the point $\textbf{x}$. The dot product has two equivalent definitions:

$$\textbf{u}\cdot\textbf{v}=u_1v_1+u_2v_2+...+u_nv_n$$ or $$\textbf{u}\cdot\textbf{v}=||\textbf{u}||||\textbf{v}||cos(\theta),$$ where $\theta$ is the angle between the vectors.

Using this second definition, the fact that $\textbf{u}$ is a unit vector, and knowledge of trigonometry, we see that the directional derivative $D\textbf{u}$ at $\textbf{x}$ is bounded: $$-||\nabla f(x)||=cos(\pi)||\nabla f(x)||\leq cos(\theta)||\nabla f(x)||$$ $$\leq cos(0)||\nabla f(x)||=||\nabla f(x)||$$

Since $0\leq||\nabla f(x)||$, we see that the maximum rate of change must occur when $\theta=0$, that is, in the direction of the gradient.

As for the directional derivative consider the direction of the vector $(2,1)$. We want to know how the value of the function changes as we move in this direction at a point $\textbf{x}$. Well, for infinitesimally small units, we are moving $2$ units in the $x$ direction and $1$ unit in the $y$ direction so the change in the function is $2$ times the change in $f$ that we get as we move $1$ unit in the $x$ direction plus $1$ times the change in $f$ that we get as we move $1$ unit in the $y$ direction. Finally, we divide by the norm of $(2,1)$ so that we have the change for $1$ unit in this direction.

• Thank you for your answer =) I changed the grad to $\nabla$ =) – jjepsuomi Apr 19 '13 at 8:01
• You're welcome and thanks for the edit. It looks better with the nabla. – jim Apr 19 '13 at 8:13

As for why the gradient points in the direction of maximum increase, let's say we don't know this and we want to find a unit vector $\vec{u}$ such that the directional derivative of some function $f$ is the greatest in the direction of $\vec{u}$. Then if $\theta$ is the angle between $\nabla f$ and $\vec{u}$ we have $\nabla f_{\vec{u}}=\nabla f \cdot \vec{u}=|\nabla f||\vec{u}|\cos(\theta)=|\nabla f|\cos(\theta)$ since $\vec{u}$ is a unit vector. This quantity is then maximized when $\cos(\theta)=1$, i.e., when $\theta=0$, thus $\vec{u}$ points in the direction of the gradient.

• Thank you very much! What about the formula of directional derivative? Why the formula is the way it is? Can you help me see it intuitively? =) – jjepsuomi Apr 19 '13 at 7:28
• Think of the usual partial derivatives as derivatives in the direction of the coordinate axes, and notice that for example $\frac{\partial f}{\partial x}=\nabla f \cdot (1,\ldots,0)$. This should guide your intuition for the general case. – dezign Apr 19 '13 at 7:35
• Thank you for your answer! +1 – jjepsuomi Apr 19 '13 at 7:46

Although the previous 2 answers are the "real" answers to your inquiry I feel that perhaps you were asking for an intuitive motivation for why grad f represents the maximum increase. The reason lies in the formal reasoning above... Think about what theta being zero really says... that grad f points in the direction orthogonal to (perpendicular) to the contour lines.. higher regions will have contour lines that get closer together and more dense and if grad f if orthogonal to these lines it must be pointing straight up.

• Thank you for your answer! =) +1 – jjepsuomi Apr 20 '13 at 7:35