3
$\begingroup$

Let $x \in \mathbb{R}^k$, $D \in \mathbb{R}^{k \times k}$, and $b \in \mathbb{R}$. Assume $D$ is symmetric. Consider the function $f : \mathbb{R}^k \rightarrow \mathbb{R}$ defined by $$f(x) = (x^TDx - b^2)^2.$$ I want to compute the Hessian $\nabla^2f$. The gradient can be computed via the chain rule: \begin{align*} \nabla f(x) & = 2(x^TDx - b^2)Dx. \\ & = 2[(x^TDx)Dx - b^2Dx].\end{align*} If we differentiate the second term again, we get $\nabla(b^2Dx) = b^2D$. However, I am not sure how to differentiate the first term. Can anyone help with this?

$\endgroup$
  • $\begingroup$ Shouldn't be \begin{align*} \nabla f(x) & = 4(x^TDx - b^2)Dx. \\ & = 4[(x^TDx)Dx - b^2Dx].\end{align*}. ? $\endgroup$ – Red shoes Jun 16 '17 at 7:14
1
$\begingroup$

Let $F(x) = (x^TDx)Dx $ then $F: R^k \to R^k.$ To find the derivative of $F$ we only need to find the gredient of each component (row) of $F$. Here is the gradient of first component of $F$ i.e., $$ F_1 (x) = (x^TDx ) \langle d^1 , x \rangle $$

Where $d^1$ is the first row of $D$. Therefore

$$ \nabla F_1 (x) = 2\langle d^1 , x \rangle Dx + (x^TDx ) d^1 $$

$\endgroup$
1
$\begingroup$

Let's do the calculation in coordinates. Write $D = (d_{ij})$. Then

$$ f(x) = \left( \sum_{i,j=1}^n d_{ij} x_i x_j - b^2 \right)^2. $$

We have

$$ \frac{\partial f}{\partial x_k} = 2 \left( \sum_{i,j=1}^n d_{ij} x_i x_j - b^2 \right) \left( \sum_{j=1}^n d_{kj} x_j\right) $$

and

$$ \frac{\partial^2 f}{\partial x_l x_k} = 2 \left( \left( \sum_{j = 1}^n d_{lj} x_j \right)\left( \sum_{j=1}^n d_{kj} x_j \right) + \left( \sum_{i,j=1}^n d_{ij} x_i x_j - b^2 \right) d_{kl}\right).$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.