I have seen the claim that the directional derivative of $f$ in the direction $\hat{v}$, where $||\hat{v}|| = 1$, (denoted $\nabla_\hat{v} f$) is equal to the gradient of $\nabla f$ dotted with $\hat{v}$.

I have tried to prove this to myself, but I got stuck: $\newcommand{\R}{\mathbb{R}}$ $\newcommand{\limit}[2]{\lim_{#1 \to #2}}$ $\newcommand{\pderiv}[2]{\dfrac{\partial#1}{\partial#2}}$

Let $f : \R^n \to \R$

I accept that

$$ \nabla_{\hat{v}} f = \limit{h}{0} \frac{f(x + h\hat{v}) - f(x)}{h}$$

for making intuitive sense. Furthermore I know that

$$ \pderiv{f}{x_i} = \limit{h}{0} \frac{f(x + h\hat{i}) - f(x)}{h} $$

where $\hat{i}$ is the unit vector of the $i$-th dimension. I also know that

$$\nabla f = \left( \pderiv{f}{x_1}, \dots, \pderiv{f}{x_n} \right)$$

Now I want to show that $\nabla_\hat{v} f = \nabla f \bullet \hat{v}$:

\begin{align*} \nabla f \bullet \hat{v} & = \left( \pderiv{f}{x_1}, \dots, \pderiv{f}{x_n} \right) \bullet v\hat{v}\\ &= \sum_{i = 1}^{n} \pderiv{f}{x_i} \cdot \hat{v}_i\\ &= \sum_{i = 1}^{n} \limit{h}{0} \frac{f(x + h\hat{i}) - f(x)}{h} \cdot \hat{v}_i\\ &= \sum_{i = 1}^{n} \limit{h}{0} \frac{\hat{v}_if(x + h\hat{i}) - \hat{v}_if(x)}{h}\\ \end{align*}

And now I am stuck. I don't see a way to transform the last line into $\limit{h}{0} \frac{f(x + h\hat{v}) - f(x)}{h}$ to reach $\nabla_{\hat{v}} f$

Can you help me out here?


I now conceptually understand why the dotproduct of direction and gradient is the directional derivative:

Let's say we have a differentiable function $f$ from $\mathbb{R}^2$ to $\mathbb{R}^3$ mapping the $xy$-plane into the $xyz$-space and we want to know the directional derivative of $f$ at a point $p = (x', y')$ for some vector $u$.

First what we need to know is, how much $f$ changes in $x$ direction and how much it changes in $y$ direction.

Then we need to realize that for small distances whatever direction we go along the surface of $f$, the total change in height is the sum of the change in height in the $x$ component of our direction and the change in height in $y$ component in our direction.

But now we can weight the partial derivatives for the $x$ and $y$ direction with the components of $u$ to get their individual contributions for the direction in which $u$ is pointing!

So if $u$ has an $x$-component of $u_x$ we weight the partial derivative of $f$ for the $x$ direction accordingly. When we do the same for $y$ we get:

$f'(x') \cdot u_x + f'(y') \cdot u_y$

which is in fact $\nabla f \bullet u$

I understood this after listeing to this lecture:


  • $\begingroup$ hint you will have to use that f is (totally) differentiable! $\endgroup$ – user412810 Mar 28 '18 at 17:35
  • $\begingroup$ Could you hint a little more? This is not an assignment or anything... I tried this proof out of my own interest. $\endgroup$ – user3578468 Mar 28 '18 at 17:39

There is more in the gradient than the $n$-tuple of partial derivatives!

The function $f:\>{\mathbb R}^n\to{\mathbb R}$ is differentiable at $x$ if there is a linear map $L:\>{\mathbb R}^n\to{\mathbb R}$ such that $$f(x+X)-f(x)=LX\ +o\bigl(|X|\bigr)\qquad(X\to0)\ .\tag{1}$$ This map is then uniquely determined, and is denoted by $df(x)$ (or similar). Since $df(x)$ in this case is a linear functional, by linear algebra there is a vector $a\in{\mathbb R}^n$ such that $$df(x).X= a\cdot X\qquad(X\in{\mathbb R}^n)\ .$$ This vector $a$ is called the gradient of $f$ at $x$, and is denoted by $\nabla f(x)$. Coordinatewise we have $$\nabla f(x)=\left({\partial f\over\partial x_1},\ldots,{\partial f\over\partial x_n}\right)_x\ ,$$ but we shall not need this. Anyway, we now can write $(1)$ in the form $$f(x+X)-f(x)=\nabla f(x)\cdot X\ +o\bigl(|X|\bigr)\qquad(X\to0)\ .\tag{2}$$ Now let a unit vector $e$ be given. Letting $X:=t\,e$ in $(2)$ implies $$f(x+ t e)-f(x)=t\>\nabla f(x)\cdot e +\ o\bigl(|t|\bigr)\qquad(t\to0)\ ,$$ hence $$D_e f(x):=\lim_{t\to0+}{f(x+ t e)-f(x)\over t}=\nabla f(x)\cdot e\ ,$$ as claimed.


Let $v:=(v_1,v_2,...,v_n)$ and $x:=(x_1,...,x_n)$. Define $y$ with $y_k:=x_k+hv_k$ for $h>0$ and $k=1,...,n$ so that $y=x+hv$. Then $$\frac{df(y_1,...,y_n)}{dh}=\frac{\partial f}{\partial y_1}\frac{dy_1}{dh}+...+\frac{\partial f}{\partial y_n}\frac{dy_n}{dh}=\frac{\partial f}{\partial y_1}v_1+...+\frac{\partial f}{\partial y_n}v_n=\langle\nabla_y f,v\rangle$$ On the other hand $$\frac{df(y_1,...,y_n)}{dh}=\frac{df(x+hv)}{dh}:=g'(h)$$ where $g(h):=f(x+hv)$. Therefore $$g'(0)=\lim_{h\to 0}\frac{f(x+hv)-f(x)}{h}=\frac{df}{dh}\Big|_{h=0}=\langle\nabla_y f,v\rangle\Big|_{h=0}=\langle\nabla_x f,v\rangle$$

  • $\begingroup$ Why is the introduction of $g$ necessary? And what does $\langle\nabla_y f,v\rangle\Big|_{h=0}$ denote? $\endgroup$ – user3578468 Mar 28 '18 at 18:08
  • $\begingroup$ $g$ is to make exposition easier since at the end what you really want is $g'(0)$. The other thing you asked follows from the first line since $y=x+hv$. $\endgroup$ – Arian Mar 28 '18 at 18:23

$\newcommand{\limit}[2]{\lim_{#1 \to #2}}$

Since this questions does not have an accepted answer yet and I found myself looking for a clear and easily understandable proof with low prerequisites. I found one in 1 which I would like to share with you.

We are trying to proof

$$ \nabla_{\boldsymbol{v}} f(\boldsymbol{x}) = \nabla_{\boldsymbol{x}} f({\boldsymbol{x}}) \cdot \boldsymbol{v}. $$

Define $\boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}: \mathbb{R} \rightarrow \mathbb{R}^{\mathrm{dim}(\boldsymbol{x})} , \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}} : h \mapsto \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}(h) := \boldsymbol{x} + h \boldsymbol{v}$

and $g_{\boldsymbol{x}, \boldsymbol{v}}: \mathbb{R}\rightarrow \mathbb{R}, g_{\boldsymbol{x}, \boldsymbol{v}} : h \mapsto g_{\boldsymbol{x}, \boldsymbol{v}}(h) := f( \boldsymbol{x} + h \boldsymbol{v} ) = f(\boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}})$

Starting from the definition of the directional derivative,

\begin{align} \nabla_{\boldsymbol{v}} f(\boldsymbol{x}) =& \limit{h}{0} \frac{f(\boldsymbol{x} + h\boldsymbol{v}) - f(\boldsymbol{x})}{h} \\ =& \limit{h}{0} \frac{g_{\boldsymbol{x}, \boldsymbol{v}}(h) - g_{\boldsymbol{x}, \boldsymbol{v}}(0)}{h} \\ =& g_{\boldsymbol{x}, \boldsymbol{v}}'(0) \equiv \frac{\mathrm{d} g_{\boldsymbol{x}, \boldsymbol{v}}} {\mathrm{d} h} \Biggr|_{0} \end{align}

Now consider for an arbitrary $h \in \mathbb{R}$

\begin{align} \frac{\mathrm{d} g_{\boldsymbol{x}, \boldsymbol{v}}(h)} {\mathrm{d} h} =& \frac{\mathrm{d} f\big( \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}(h) \big)} {\mathrm{d} h} \\ =& \frac{\mathrm{d} f} {\mathrm{d} \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}} \cdot \frac{\mathrm{d} \boldsymbol{y}_{\boldsymbol{x}, \boldsymbol{v}}}{\mathrm{d}h} \\ =& \nabla_{\boldsymbol{y}_{\boldsymbol{x}, \boldsymbol{v}}} f \cdot \frac{\mathrm{d} (\boldsymbol{x} + h \boldsymbol{v})}{\mathrm{d}h} \\ =& \nabla_{(\boldsymbol{x} + h \boldsymbol{v})} f \cdot \boldsymbol{v} \end{align}

Which gives for $h=0$

$$ \nabla_{\boldsymbol{v}} f(\boldsymbol{x}) = g_{\boldsymbol{x}, \boldsymbol{v}}'(0) \equiv \frac{\mathrm{d} g_{\boldsymbol{x}, \boldsymbol{v}}} {\mathrm{d} h} \Biggr|_{0} = \nabla_{\boldsymbol{x} } f \cdot \boldsymbol{v} .$$


1 Jorge Nocedal, Stephen J. Wright. Numerical Optimization. Springer, New York, NY. Proof can be found on p. 581 of the second edition. DOI. URL Print ISBN: 978-0-387-98793-4. Online ISBN: 978-0-387-22742-9


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.