# Is the Hessian Equal to the Outer Product of the Score with Itself

Recall for a function $f: \mathbb{R}^n \to \mathbb{R}$, the Gradient is $$\nabla f(\mathbf{x}) = \begin{bmatrix} \frac{\partial f(\mathbf{x})}{\partial x_1} & \frac{\partial f(\mathbf{x})}{\partial x_2} & \cdots & \frac{\partial f(\mathbf{x})}{\partial x_n} \end{bmatrix}$$ and the Hessian is $$Hf(\mathbf{x}) = \begin{bmatrix} \frac{\partial^2 f(\mathbf{x})}{(\partial x_1)^2} & \frac{\partial^2 f(\mathbf{x})}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2 f(\mathbf{x})}{\partial x_1 \partial x_n} \\ \frac{\partial^2 f(\mathbf{x})}{\partial x_2\partial x_1} & \frac{\partial^2 f(\mathbf{x})}{(\partial x_2)^2} & \cdots & \frac{\partial^2 f(\mathbf{x})}{\partial x_2 \partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f(\mathbf{x})}{\partial x_n\partial x_1} & \frac{\partial^2 f(\mathbf{x})}{\partial x_n\partial x_2} & \cdots & \frac{\partial^2 f(\mathbf{x})}{(\partial x_n)^2} \end{bmatrix}$$ To me, this looks like an outer product matrix. So, this brings me to my question: given the gradient of $f$, if I wanted to compute the Hessian, is it true that \begin{align} Hf(\mathbf{x})&= \begin{bmatrix} \frac{\partial f(\mathbf{x})}{\partial x_1} \\ \frac{\partial f(\mathbf{x})}{\partial x_2} \\ \vdots \\ \frac{\partial f(\mathbf{x})}{\partial x_n} \end{bmatrix} \begin{bmatrix} \frac{\partial f(\mathbf{x})}{\partial x_1} & \frac{\partial f(\mathbf{x})}{\partial x_2} & \cdots & \frac{\partial f(\mathbf{x})}{\partial x_n} \end{bmatrix} \\ &=\nabla f(\mathbf{x})^{\top}\nabla f(\mathbf{x}) \\ &=sf(\mathbf{x})sf(\mathbf{x})^{\top} \end{align} where $sf(\mathbf{x})=\nabla f(\mathbf{x})^{\top}$ denotes the score? Thank you!

• It would be more correct to write $H=\nabla^T\nabla$, where the differentiation operator themselves would be multiplied as opposed to their applications to $f$. – Solomonoff's Secret Nov 10 '14 at 17:17
• Aha! Okay, thanks. Is it also true that the Hessian is equal to the Jacobian of the score, so that $Hf(\mathbf{x})=\nabla sf(\mathbf{x})$? – Mathemanic Nov 12 '14 at 6:54
• The Hessian is the Jacobian of the Gradient, but the Jacobian is normally written as $J$. See en.wikipedia.org/wiki/Hessian_matrix . – Solomonoff's Secret Nov 12 '14 at 18:05

$\frac{\partial^2 f}{\partial x_1 \partial x_2}(\mathbf{x}) \neq \frac{\partial f}{\partial x_1}(\mathbf{x}) \cdot \frac{\partial f}{\partial x_2}(\mathbf{x})$
• Thanks for pointing this out! For clarification, is the Hessian equal to the Jacobian of the score, so that $Hf(\mathbf{x})=\nabla sf(\mathbf{x})$? – Mathemanic Nov 12 '14 at 6:53
$\frac{\partial^2 f(\mathbf{x})}{\partial x_2 \partial x_1} \neq (\frac{\partial f(\mathbf{x})}{\partial x_2})*(\frac{\partial f(\mathbf{x})}{\partial x_1})$