1
$\begingroup$

Let a $\theta$ be a 2 Dimensional vector and $\alpha$, $\beta$ be positive scalars. Let $f(\theta)$ be a function given as:

$ f(\theta) = \alpha \exp(-\beta \: \theta^T \theta)$

I can find the gradient of the function using the chain rule but I am having some difficulty in finding the Hessian. The gradient which I computed is the following:

$\nabla f(\theta) = - 2 \alpha \beta \exp(-\beta \: \theta^T \theta) \:\theta$

What is the Hessian of $f(\theta)$. In other words what is the derivative of $\nabla f(\theta)$.

$\endgroup$
1
$\begingroup$

If $\theta = (x,y)$, then $$f(x,y) = \alpha e^{-\beta(x^2+y^2)}. $$ So $f_x(x,y) = -2\alpha \beta x e^{-\beta(x^2+y^2)}$, and so $$f_{xx}(x,y) =-2\alpha \beta e^{-\beta(x^2+y^2)} +4\alpha \beta^2 x^2 e^{-\beta(x^2+y^2)} = 2\alpha\beta (2\beta x^2-1)e^{-\beta (x^2+y^2)}.$$By symmetry: $$f_{yy}(x,y) = 2\alpha\beta (2\beta y^2-1)e^{-\beta (x^2+y^2)}.$$ Finally: $$f_{xy}(x,y)=4\alpha \beta^2 xye^{-\beta (x^2+y^2)}.$$So we conclude that $${\rm Hess}\;f (x,y) =2\alpha\beta e^{-\beta (x^2+y^2)}\begin{pmatrix} 2\beta x^2-1 & 2\beta xy \\ 2\beta xy & 2\beta y^2-1 \end{pmatrix}. $$Going back to $\theta$ notation, this writes as $${\rm Hess}\;f (\theta) = 2\alpha\beta e^{-\beta \theta^\top\theta} (2\beta \theta \theta^\top -{\rm Id}_2). $$

$\endgroup$
  • $\begingroup$ Is it possible to derive this for an n-dimensional $\theta$ using vector calculus? $\endgroup$ – neo89 Oct 3 '17 at 15:12
  • $\begingroup$ The results are the same if $\theta\in \Bbb R^n$. I actually tried to understand how to do it without using components, you might find it helpful too. $\endgroup$ – Ivo Terek Oct 3 '17 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.