1
$\begingroup$

Let $x\in \mathbb{R}^n$ and $A\in\mathbb{R}^{n\times n}$ be a positive semi-definite matrix.

Is there a way to express in closed form the following derivative?

$$\frac{\partial}{\partial x} \sqrt{x^TAx}$$

$\endgroup$
3
  • $\begingroup$ In this context, are positive semidefinite matrices necessarily symmetric? $\endgroup$ Dec 7, 2020 at 15:47
  • 1
    $\begingroup$ In any case, the answer will be $\frac{1}{2\sqrt{x^TAx}}(A + A^T)x$. $\endgroup$ Dec 7, 2020 at 15:48
  • $\begingroup$ Thanks, Ben. If you post this as an answer I can close this thread and confirm the answer. It would be nice also if you can add some more information on the derivation. $\endgroup$
    – Apprentice
    Dec 7, 2020 at 15:51

3 Answers 3

4
$\begingroup$

First of all, using the chain rule, we have $$ \frac{\partial }{\partial x} \sqrt{x^TAx} = \frac{1}{2 \sqrt{x^TAx}}\cdot \frac{\partial }{\partial x} x^TAx. $$ One approach to this partial derivative is to write the expression $f(x + h)$ in the form $f(x) + g(x)^Th + o(h)$; by definition, the $g(x)$ for which this holds is equal to $\frac{\partial f}{\partial x}$. With that in mind, $$ (x+h)^TA(x + h) = x^TAx + x^TAh + h^TAx + h^TAh\\ = x^TAx + x^TAh + [h^TAx]^T + o(h)\\ = x^TAx + x^TAh + x^TA^Th + o(h)\\ = x^TAx + [(A + A^T)x]^Th + o(h). $$ With that, the derivative of $x \mapsto x^TAx$ is $(A + A^Tx)$, and we have $$ \frac{\partial }{\partial x} \sqrt{x^TAx} = \frac{1}{2 \sqrt{x^TAx}}\cdot (A + A^T)x. $$

$\endgroup$
2
$\begingroup$

$ \def\o{{\tt1}}\def\p{\partial} \def\L{\left}\def\R{\right}\def\LR#1{\L(#1\R)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\grad#1#2{\frac{\p #1}{\p #2}} $The simplest approach is to square the function and then use implicit differentiation $$\eqalign{ f^2 &= x^TAx \\ 2f\;df &= \LR{dx^TAx+x^TA\,dx} = \LR{Ax+A^Tx}^Tdx \\ \grad{f}{x} &= \frac{\LR{A+A^T}x}{2f} = \frac{\LR{A+A^T}x}{2\sqrt{x^TAx}} \\ }$$

$\endgroup$
1
$\begingroup$

If $A$ is symmetric, you can rewrite it as $$A = B^T B$$ for some matrix $B$ (see more information on how to find $B$ here).

Hence we have $$x^TAx = x^TB^TBx = (Bx)^TBx = \|Bx\|^2$$

Taking the square root and differentiating should yield a result.
Where naturally $\|Bx\|^2$ is a scalar.

Hope that helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.