# Second directional derivative and Hessian matrix: a clarification on the proof

I have been reading this answer and I couldn't get the proof.

The argument is about how we can write the second directional derivative of $f:\mathbf{R}^m\rightarrow\mathbf{R}$ with respect to direction $u$. The argument goes as follow:

First directional derivative of $f:\mathbf{R}^m\rightarrow \mathbf{R}$ in the direction of $u$ at $x$ is given by $$\partial_u f(x):=\lim_{t\rightarrow 0}\frac{f(x+tu)-f(x)}{t}=\nabla f(x) \cdot u = \sum_{i=1}^{m} u_i\partial_{x_i}f(x). \label{}$$ The second directional derivative along the direction $u$ is given in the similar fasion: \begin{align*} \partial^2_{uu}f(x)&=\partial_u(\partial_u f)\\ &=\lim_{t\rightarrow 0}\frac{\partial_u f(x+tu)-\partial_u f(x)}{t}\\ &=\lim_{t\rightarrow 0}\frac{\nabla f(x+tu)\cdot u-\nabla f(x)\cdot u}{t}\\ &=\lim_{t\rightarrow 0}\frac{u_i \partial_{x_i}f(x+tu)-u_i \partial_{x_i}f(x)}{t}\\ &=u_i \partial_{x_i x_j} f(x)u_j\\ &=u^THu \label{} \end{align*} where $H=D^2 f(x)$ is the Hessian matrix of $f$ at $x$.

1. Is $$\lim_{t\rightarrow 0}\frac{u_i \partial_{x_i}f(x+tu)-u_i \partial_{x_i}f(x)}{t}$$ the same as $$\lim_{t\rightarrow 0}\frac{\sum\limits_{i=0}^m u_i \partial_{x_i}f(x+tu) - \sum\limits_{j=0}^m u_j \partial_{x_j}f(x)}{t}\\$$?

2. If so, how we can get to $$u_i \partial_{x_i x_j} f(x)u_j$$?

I just can't see it

The answer to question one use yes $$u_i\partial_{x_i}f(x+tu) = \sum_{i=0}^m u_i\partial_{x_i}f(x+tu)$$ and it holds true even for the second factor. This is called Einstein notation, mainly: repeated indices are to be summed over.
For the second question $$\lim_{t\rightarrow 0}\frac{u_i\partial_{x_i}f(x+tu)-u_i\partial_{x_i}f(x)}{t} = u_i\underbrace{\lim_{t\rightarrow 0}\frac{\partial_{x_i}f(x+tu)-\partial_{x_i}f(x)}{t}}_{\text{dir derivative of }\partial_{x_i}f(x) \text{ along }u}$$ The second limit, being the directional derivative of the function $\partial_{x_i}f(x)$ evaluates, with some abuse of notation, to $$\nabla (\partial_{x_i}f(x))\cdot u = \partial_{x_j}(\partial_{x_i}f(x))u_j = \partial_{x_ix_j}f(x)u_j$$ where I expanded the scalar product using Einstein notation. Now plugging it back into the limit we get $$u_i\lim_{t\rightarrow 0}\frac{\partial_{x_i}f(x+tu)-\partial_{x_i}f(x)}{t} = u_i\partial_{x_ix_j}f(x)u_j$$
• Thanks. I have difficulties in following the Einstein notation, but by making an example with $m=2$ it is clear (in particular I didn't understand why you took $u_i$ out if the indexes of the summation are different, one is over $i$ and the other is over $j$, but now it is clear) – rtrtrt Aug 3 '18 at 20:36