Given a function $f: \mathbb{R}^m \to \mathbb{R}$, why does $\mathbf{d}^{t} H \mathbf{d}$ give the second derivative to $f$ along the unit vector $\mathbf{d}$? Here $H$ is the Hessian, so that $H_{ij} = \frac{\partial f}{\partial x_i \partial x_j}$. I suppose I don't have the right definition to work with, so any intuition would be helpful, as well as formal proof. (e.g. why this works in the second order taylor series expansion)
$\begingroup$
$\endgroup$
asked Jun 30 '17 at 22:44
-
$\begingroup$ I don't think there is anything wrong with your definition, you could actually just carry out the matrix multiplication and use some examples like the basis vectors $e_i$. $\endgroup$ – Triatticus Jun 30 '17 at 23:01
-
$\begingroup$ @Triatticus could you carry out one such calculation? $\endgroup$ – Anthony Peter Jun 30 '17 at 23:03
-
$\begingroup$ I believe it would be something like $\sum_i \sum_j H_{ij}d_i d_j$. The for example if d is the first unit vector $e_1$ this means $d_i=d_j=1$ for $i=j=1$ and zero otherwise, this yields $H_{11}d_1d_1=H_{11} = \frac{\partial^2}{\partial x_1 \partial x_1}$ $\endgroup$ – Triatticus Jun 30 '17 at 23:05
-
$\begingroup$ @Triatticus yes i agree. but why is this the right second derivative? $\endgroup$ – Anthony Peter Jun 30 '17 at 23:06
-
1$\begingroup$ @AnthonyPeter alternatively: can we start from the assumption that this makes sense when $d$ is taken from the standard basis, e.g. $d = (1,0,\dots,0)$? $\endgroup$ – Ben Grossmann Jun 30 '17 at 23:14
$\begingroup$
$\endgroup$
Consider the function of one variables given by $g(t)=f(x+td)$. You want to compute $g''(0)$. So you need to use the chain rule. You get $g'(t)=\sum_i\frac{\partial f}{\partial x_i}(x+td)d_i$ and
$g''(t)=\sum_i\sum_j\frac{\partial^2 f}{\partial x_i\partial x_j}(x+td)d_i d_j$. So
$g''(0)=\sum_i\sum_j\frac{\partial^2 f}{\partial x_i\partial x_j}(x)d_i d_j= d^tH(x)d$.
