# Why is the dotproduct of direction and gradient the directional derivative? ($\nabla_\hat{v} f = \nabla f \bullet \hat{v}$)

I have seen the claim that the directional derivative of $$f$$ in the direction $$\hat{v}$$, where $$||\hat{v}|| = 1$$, (denoted $$\nabla_\hat{v} f$$) is equal to the gradient of $$\nabla f$$ dotted with $$\hat{v}$$.

I have tried to prove this to myself, but I got stuck: $$\newcommand{\R}{\mathbb{R}}$$ $$\newcommand{\limit}[2]{\lim_{#1 \to #2}}$$ $$\newcommand{\pderiv}[2]{\dfrac{\partial#1}{\partial#2}}$$

Let $$f : \R^n \to \R$$

I accept that

$$\nabla_{\hat{v}} f = \limit{h}{0} \frac{f(x + h\hat{v}) - f(x)}{h}$$

for making intuitive sense. Furthermore I know that

$$\pderiv{f}{x_i} = \limit{h}{0} \frac{f(x + h\hat{i}) - f(x)}{h}$$

where $$\hat{i}$$ is the unit vector of the $$i$$-th dimension. I also know that

$$\nabla f = \left( \pderiv{f}{x_1}, \dots, \pderiv{f}{x_n} \right)$$

Now I want to show that $$\nabla_\hat{v} f = \nabla f \bullet \hat{v}$$:

\begin{align*} \nabla f \bullet \hat{v} & = \left( \pderiv{f}{x_1}, \dots, \pderiv{f}{x_n} \right) \bullet v\hat{v}\\ &= \sum_{i = 1}^{n} \pderiv{f}{x_i} \cdot \hat{v}_i\\ &= \sum_{i = 1}^{n} \limit{h}{0} \frac{f(x + h\hat{i}) - f(x)}{h} \cdot \hat{v}_i\\ &= \sum_{i = 1}^{n} \limit{h}{0} \frac{\hat{v}_if(x + h\hat{i}) - \hat{v}_if(x)}{h}\\ \end{align*}

And now I am stuck. I don't see a way to transform the last line into $$\limit{h}{0} \frac{f(x + h\hat{v}) - f(x)}{h}$$ to reach $$\nabla_{\hat{v}} f$$

Can you help me out here?

# Edit

I now conceptually understand why the dotproduct of direction and gradient is the directional derivative:

Let's say we have a differentiable function $$f$$ from $$\mathbb{R}^2$$ to $$\mathbb{R}^3$$ mapping the $$xy$$-plane into the $$xyz$$-space and we want to know the directional derivative of $$f$$ at a point $$p = (x', y')$$ for some vector $$u$$.

First what we need to know is, how much $$f$$ changes in $$x$$ direction and how much it changes in $$y$$ direction.

Then we need to realize that for small distances whatever direction we go along the surface of $$f$$, the total change in height is the sum of the change in height in the $$x$$ component of our direction and the change in height in $$y$$ component in our direction.

But now we can weight the partial derivatives for the $$x$$ and $$y$$ direction with the components of $$u$$ to get their individual contributions for the direction in which $$u$$ is pointing!

So if $$u$$ has an $$x$$-component of $$u_x$$ we weight the partial derivative of $$f$$ for the $$x$$ direction accordingly. When we do the same for $$y$$ we get:

$$f'(x') \cdot u_x + f'(y') \cdot u_y$$

which is in fact $$\nabla f \bullet u$$

I understood this after listeing to this lecture:

• hint you will have to use that f is (totally) differentiable!
– user412810
Mar 28, 2018 at 17:35
• Could you hint a little more? This is not an assignment or anything... I tried this proof out of my own interest. Mar 28, 2018 at 17:39

There is more in the gradient than the $n$-tuple of partial derivatives!

The function $f:\>{\mathbb R}^n\to{\mathbb R}$ is differentiable at $x$ if there is a linear map $L:\>{\mathbb R}^n\to{\mathbb R}$ such that $$f(x+X)-f(x)=LX\ +o\bigl(|X|\bigr)\qquad(X\to0)\ .\tag{1}$$ This map is then uniquely determined, and is denoted by $df(x)$ (or similar). Since $df(x)$ in this case is a linear functional, by linear algebra there is a vector $a\in{\mathbb R}^n$ such that $$df(x).X= a\cdot X\qquad(X\in{\mathbb R}^n)\ .$$ This vector $a$ is called the gradient of $f$ at $x$, and is denoted by $\nabla f(x)$. Coordinatewise we have $$\nabla f(x)=\left({\partial f\over\partial x_1},\ldots,{\partial f\over\partial x_n}\right)_x\ ,$$ but we shall not need this. Anyway, we now can write $(1)$ in the form $$f(x+X)-f(x)=\nabla f(x)\cdot X\ +o\bigl(|X|\bigr)\qquad(X\to0)\ .\tag{2}$$ Now let a unit vector $e$ be given. Letting $X:=t\,e$ in $(2)$ implies $$f(x+ t e)-f(x)=t\>\nabla f(x)\cdot e +\ o\bigl(|t|\bigr)\qquad(t\to0)\ ,$$ hence $$D_e f(x):=\lim_{t\to0+}{f(x+ t e)-f(x)\over t}=\nabla f(x)\cdot e\ ,$$ as claimed.

$$\newcommand{\limit}[2]{\lim_{#1 \to #2}}$$ Since this questions does not have an accepted answer yet and I found myself looking for a clear and easily understandable proof with low prerequisites. I found one in 1 which I would like to share with you.

We are trying to prove

$$\nabla_{\boldsymbol{v}} f(\boldsymbol{x}) = \nabla_{\boldsymbol{x}} f({\boldsymbol{x}}) \cdot \boldsymbol{v}.$$

Define $$\boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}: \mathbb{R} \rightarrow \mathbb{R}^{\mathrm{dim}(\boldsymbol{x})} , \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}} : h \mapsto \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}(h) := \boldsymbol{x} + h \boldsymbol{v}$$

and $$g_{\boldsymbol{x}, \boldsymbol{v}}: \mathbb{R}\rightarrow \mathbb{R}, g_{\boldsymbol{x}, \boldsymbol{v}} : h \mapsto g_{\boldsymbol{x}, \boldsymbol{v}}(h) := f( \boldsymbol{x} + h \boldsymbol{v} ) = f(\boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}})$$

Starting from the definition of the directional derivative,

\begin{align} \nabla_{\boldsymbol{v}} f(\boldsymbol{x}) =& \limit{h}{0} \frac{f(\boldsymbol{x} + h\boldsymbol{v}) - f(\boldsymbol{x})}{h} \\ =& \limit{h}{0} \frac{g_{\boldsymbol{x}, \boldsymbol{v}}(h) - g_{\boldsymbol{x}, \boldsymbol{v}}(0)}{h} \\ =& g_{\boldsymbol{x}, \boldsymbol{v}}'(0) \equiv \frac{\mathrm{d} g_{\boldsymbol{x}, \boldsymbol{v}}} {\mathrm{d} h} \Biggr|_{0} \end{align}

Now consider for an arbitrary $$h \in \mathbb{R}$$

\begin{align} \frac{\mathrm{d} g_{\boldsymbol{x}, \boldsymbol{v}}(h)} {\mathrm{d} h} =& \frac{\mathrm{d} f\big( \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}(h) \big)} {\mathrm{d} h} \\ =& \frac{\mathrm{d} f} {\mathrm{d} \boldsymbol{y_{\boldsymbol{x}, \boldsymbol{v}}}} \cdot \frac{\mathrm{d} \boldsymbol{y}_{\boldsymbol{x}, \boldsymbol{v}}}{\mathrm{d}h} \\ =& \nabla_{\boldsymbol{y}_{\boldsymbol{x}, \boldsymbol{v}}} f \cdot \frac{\mathrm{d} (\boldsymbol{x} + h \boldsymbol{v})}{\mathrm{d}h} \\ =& \nabla_{(\boldsymbol{x} + h \boldsymbol{v})} f \cdot \boldsymbol{v} \end{align}

Which gives for $$h=0$$

$$\nabla_{\boldsymbol{v}} f(\boldsymbol{x}) = g_{\boldsymbol{x}, \boldsymbol{v}}'(0) \equiv \frac{\mathrm{d} g_{\boldsymbol{x}, \boldsymbol{v}}} {\mathrm{d} h} \Biggr|_{0} = \nabla_{\boldsymbol{x} } f \cdot \boldsymbol{v} .$$

References:

1 Jorge Nocedal, Stephen J. Wright. Numerical Optimization. Springer, New York, NY. Proof can be found on p. 581 of the second edition. DOI. URL Print ISBN: 978-0-387-98793-4. Online ISBN: 978-0-387-22742-9

Let $v:=(v_1,v_2,...,v_n)$ and $x:=(x_1,...,x_n)$. Define $y$ with $y_k:=x_k+hv_k$ for $h>0$ and $k=1,...,n$ so that $y=x+hv$. Then $$\frac{df(y_1,...,y_n)}{dh}=\frac{\partial f}{\partial y_1}\frac{dy_1}{dh}+...+\frac{\partial f}{\partial y_n}\frac{dy_n}{dh}=\frac{\partial f}{\partial y_1}v_1+...+\frac{\partial f}{\partial y_n}v_n=\langle\nabla_y f,v\rangle$$ On the other hand $$\frac{df(y_1,...,y_n)}{dh}=\frac{df(x+hv)}{dh}:=g'(h)$$ where $g(h):=f(x+hv)$. Therefore $$g'(0)=\lim_{h\to 0}\frac{f(x+hv)-f(x)}{h}=\frac{df}{dh}\Big|_{h=0}=\langle\nabla_y f,v\rangle\Big|_{h=0}=\langle\nabla_x f,v\rangle$$

• Why is the introduction of $g$ necessary? And what does $\langle\nabla_y f,v\rangle\Big|_{h=0}$ denote? Mar 28, 2018 at 18:08
• $g$ is to make exposition easier since at the end what you really want is $g'(0)$. The other thing you asked follows from the first line since $y=x+hv$. Mar 28, 2018 at 18:23