Derivative of $\sqrt{x^TAx}$ Let $x\in \mathbb{R}^n$ and $A\in\mathbb{R}^{n\times n}$ be a positive semi-definite matrix.
Is there a way to express in closed form the following derivative?
$$\frac{\partial}{\partial x} \sqrt{x^TAx}$$
 A: First of all, using the chain rule, we have
$$
\frac{\partial }{\partial x} \sqrt{x^TAx} = \frac{1}{2 \sqrt{x^TAx}}\cdot  \frac{\partial }{\partial x} x^TAx.
$$
One approach to this partial derivative is to write the expression $f(x + h)$ in the form $f(x) + g(x)^Th + o(h)$; by definition, the $g(x)$ for which this holds is equal to $\frac{\partial f}{\partial x}$. With that in mind,
$$
(x+h)^TA(x + h) = x^TAx + x^TAh + h^TAx + h^TAh\\
= x^TAx + x^TAh + [h^TAx]^T + o(h)\\
= x^TAx + x^TAh + x^TA^Th + o(h)\\
= x^TAx + [(A + A^T)x]^Th + o(h).
$$
With that, the derivative of $x \mapsto x^TAx$ is $(A + A^Tx)$, and we have
$$
\frac{\partial }{\partial x} \sqrt{x^TAx} = \frac{1}{2 \sqrt{x^TAx}}\cdot  (A + A^T)x.
$$
A: $
\def\o{{\tt1}}\def\p{\partial}
\def\L{\left}\def\R{\right}\def\LR#1{\L(#1\R)}
\def\trace#1{\operatorname{Tr}\LR{#1}}
\def\grad#1#2{\frac{\p #1}{\p #2}}
$The simplest approach is to square the function and then use implicit differentiation
$$\eqalign{
f^2 &= x^TAx \\
2f\;df &= \LR{dx^TAx+x^TA\,dx}
  = \LR{Ax+A^Tx}^Tdx \\
\grad{f}{x} &= \frac{\LR{A+A^T}x}{2f}
  = \frac{\LR{A+A^T}x}{2\sqrt{x^TAx}} \\
}$$
A: If $A$ is symmetric, you can rewrite it as
$$A = B^T B$$
for some matrix $B$ (see more information on how to find $B$ here).
Hence we have
$$x^TAx = x^TB^TBx = (Bx)^TBx = \|Bx\|^2$$
Taking the square root and differentiating should yield a result.
Where naturally $\|Bx\|^2$ is a scalar.
Hope that helps!
