Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I need a help with one example. I have to proove that hessian matrix of a quadratic form $f(x)=x^TAx$ is $f^{\prime\prime}(x) = A + A^T$. I am not even sure how the Jaxobian looks like (I never did one for $x \in R^n$). Thanks for any advice.

share|improve this question
add comment

2 Answers 2

up vote 2 down vote accepted

So let's compute the first derivative, by definition we need to find $f'(x)\colon\mathbb R^n \to \mathbb R^n$ such that $$ f(x+h) = f(x) + f'(x)h + o(h), \qquad h \to 0 $$ We have \begin{align*} f(x+h) &= (x+h)^tA(x+h)\\ &= x^tAx + h^tAx + x^tAh + h^tAh\\ &= f(x) + x^t(A + A^t)h + h^tAh \end{align*} As $|h^tAh|\le \|A\||h|^2 = o(h)$, we have $f'(x) = x^t(A + A^t)$ for each $x \in \mathbb R^n$. Now compute $f''$, we have \begin{align*} f'(x+h) &= x^t(A + A^t) + h^t(A + A^t)\\ &= f(x) + h^t(A + A^t) \end{align*} So $f''(x) = A + A^t$.

share|improve this answer
add comment

Intuitively, the gradient and Hessian of $f$ satisfy \begin{equation} f(x + \Delta x) \approx f(x) + \nabla f(x)^T \Delta x + \frac12 \Delta x^T Hf(x) \Delta x \end{equation} and the Hessian is symmetric.

In this problem, \begin{align*} f(x + \Delta x) &= (x + \Delta x)^T A (x + \Delta x) \\ &= x^T A x + \Delta x^T A x + x^T A \Delta x + \Delta x^T A \Delta x \\ &= x^T A x + \Delta x^T(A + A^T)x + \frac12 \Delta x(A + A^T) \Delta x. \end{align*}

Comparing this with the approximate equality above, we see that $\nabla f(x) = (A + A^T) x$ and $Hf(x) = A + A^T$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.