Given a function $f: \mathbb{R}^m \to \mathbb{R}$, why does $\mathbf{d}^{t} H \mathbf{d}$ give the second derivative to $f$ along the unit vector $\mathbf{d}$? Here $H$ is the Hessian, so that $H_{ij} = \frac{\partial f}{\partial x_i \partial x_j}$. I suppose I don't have the right definition to work with, so any intuition would be helpful, as well as formal proof. (e.g. why this works in the second order taylor series expansion)

  • $\begingroup$ I don't think there is anything wrong with your definition, you could actually just carry out the matrix multiplication and use some examples like the basis vectors $e_i$. $\endgroup$ – Triatticus Jun 30 '17 at 23:01
  • $\begingroup$ @Triatticus could you carry out one such calculation? $\endgroup$ – Anthony Peter Jun 30 '17 at 23:03
  • $\begingroup$ I believe it would be something like $\sum_i \sum_j H_{ij}d_i d_j$. The for example if d is the first unit vector $e_1$ this means $d_i=d_j=1$ for $i=j=1$ and zero otherwise, this yields $H_{11}d_1d_1=H_{11} = \frac{\partial^2}{\partial x_1 \partial x_1}$ $\endgroup$ – Triatticus Jun 30 '17 at 23:05
  • $\begingroup$ @Triatticus yes i agree. but why is this the right second derivative? $\endgroup$ – Anthony Peter Jun 30 '17 at 23:06
  • 1
    $\begingroup$ @AnthonyPeter alternatively: can we start from the assumption that this makes sense when $d$ is taken from the standard basis, e.g. $d = (1,0,\dots,0)$? $\endgroup$ – Ben Grossmann Jun 30 '17 at 23:14

Consider the function of one variables given by $g(t)=f(x+td)$. You want to compute $g''(0)$. So you need to use the chain rule. You get $g'(t)=\sum_i\frac{\partial f}{\partial x_i}(x+td)d_i$ and $g''(t)=\sum_i\sum_j\frac{\partial^2 f}{\partial x_i\partial x_j}(x+td)d_i d_j$. So
$g''(0)=\sum_i\sum_j\frac{\partial^2 f}{\partial x_i\partial x_j}(x)d_i d_j= d^tH(x)d$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.