Derivative of a quadratic form using the chain rule Is there a way to calculate the derivative of a quadratic form
$$
\frac{\partial x^TAx}{\partial x} = x^T(A + A^T)
$$
using the chain rule of matrix differentiation?
$$
\frac{\partial[UV]}{\partial x} = \frac{\partial U}{\partial x} V + U\frac{\partial V}{\partial x}
$$
If not, what are the cases when the chain rule for matrix differentiation is applicable?
 A: 
Using the chain rule for matrix differentiation
  $$\frac{\partial[UV]}{\partial x} = \frac{\partial U}{\partial x}V + U\frac{\partial V}{\partial x}$$

But that is not the chain rule. That is the Leibniz (or product) rule. I assume that is what you meant. In that case the answer is yes. Here I show how to do it using index notation and Einstein summation convention.
Say our column vectors have upper indices, and our row vectors have lower indices, and our matrices have one upper and one lower index.
In this notation, matrix multiplication $AB$ is a summation over a lower index of $A$ and an upper index of $B$:
$${(AB)^{i}}_{j} = {A^{i}}_{k}{B^{k}}_{j} = {B^{k}}_{j}{A^{i}}_{k}$$
and the sum of matrices is given by
$${(A+B)^{i}}_{j} = {A^{i}}_{j} + {B^{i}}_{j}$$
Then we use the kronecker delta
$$\delta_{ij}
=\begin{cases}
0 & i \neq j \\
1 & i=j
\end{cases}$$
to define the transpose $x^{T}$ of $x$ as
$$(x^{T})_{i}=\delta_{ij}x^{j}$$
which implies
$$x^{i} = \delta^{ij}(x^{T})_{j}$$
And we define the transpose $A^{T}$ of the matrix $A$ by
$${(A^{T})^{i}}_{j} = \delta^{ik}{A^{l}}_{k}\delta_{jl}$$
Now you want
$$\frac{\partial (x^{T}Ax)}{\partial x}\tag{1}$$
which I translate to index notation
$$\left(\frac{\partial (x^{T}Ax)}{\partial x}\right)_{k} = \frac{\partial (x^{T}Ax)}{\partial x^{k}} = \frac{\partial [(x^{T})_{i}{A^{i}}_{j}x^{j}]}{\partial x^{k}}$$
and then use the product rule
$$=\frac{\partial (x^{T})_{i}}{\partial x^{k}}{A^{i}}_{j}x^{j}
+(x^{T})_{i}\frac{\partial {A^{i}}_{j}}{\partial x^{k}}x^{j}
+(x^{T})_{i}{A^{i}}_{j}\frac{\partial x^{j}}{\partial x^{k}} \tag{2}$$
The partial derivative in the first summand on the right hand side is
$$\frac{\partial (x^{T})_{i}}{\partial x^{k}}
= \frac{\partial (x^{m}\delta_{mi})}{\partial x^{k}}
= \frac{\partial x^{m}}{\partial x^{k}}\delta_{mi}
= \delta^{m}_{k}\delta_{mi}
= \delta_{ki}$$
where I used


*

*The definition of the transpose $(x^{T})_{i} = x^{m}\delta_{mi}$

*$\delta_{mi}$ is constant

*$\frac{\partial x^{m}}{\partial x^{k}} = \delta^{m}_{k}$


The second summand vanishes, because $\frac{\partial {A^{i}}_{j}}{\partial x^{k}} = 0$ since $A$ does not depend on $x$.
Using all this, equation $(2)$ reduces to
$$=\delta_{ki}{A^{i}}_{j}x^{j}+(x^{T})_{i}{A^{i}}_{j}\delta^{j}_{k}\\
=\delta_{ki}{A^{i}}_{j}\delta^{jl}(x^{T})_{l}+(x^{T})_{i}{A^{i}}_{k}\\
={(A^{T})^{l}}_{k}(x^{T})_{l}+(x^{T})_{i}{A^{i}}_{k}\\
={(A^{T})^{i}}_{k}(x^{T})_{i}+(x^{T})_{i}{A^{i}}_{k}\\
=(x^{T})_{i}\left({(A^{T})^{i}}_{k}+{A^{i}}_{k}\right)\\
=(x^{T})_{i}{\left(A^{T}+A\right)^{i}}_{k}\\
=\left(x^{T}(A^{T}+A)\right)_{k}\\
$$
Where I have used (starting from the second equal sign)


*

*$x^{i} = \delta^{ij}(x^{T})_{j}$

*The definition of the transpose ${(A^{T})^{l}}_{k} = \delta^{jl}{A^{i}}_{j}\delta_{ik}$

*Renaming the dummy index $l$ to $i$

*Factorize $(x^{T})_{i}$

*Sum of matrices

*Matrix multiplication (lower index of $x$ and upper index of $A^{T}+A$)


Hence
$$\left(\frac{\partial (x^{T}Ax)}{\partial x}\right)_{k} = \left(x^{T}(A^{T}+A)\right)_{k}$$
or, without the indices
$$\frac{\partial (x^{T}Ax)}{\partial x} = x^{T}(A^{T}+A)$$

If you want to use purely matrix notation, without the indices, I guess you could say that
$$\frac{\partial x^{T}}{\partial x}$$
is an operator that transposes the column vector immediately on its right. Then  you could take equation $(1)$ and proceed as follows
$$\frac{\partial (x^{T}Ax)}{\partial x}
= \frac{\partial x^{T}}{\partial x}Ax
+ x^{T}\frac{\partial A}{\partial x}x
+ x^{T}A\frac{\partial x}{\partial x}\\
= (Ax)^{T} + x^{T}A\\
= x^{T}A^{T} + x^{T}A\\
= x^{T}(A^{T} + A)$$
But one (at least I) wouldn't have seen and understood this behaviour of $\frac{\partial x^{T}}{\partial x}$ were it not for the calculation in index notation.

I'll have to rectify: there is a way to see that behaviour of $\frac{\partial x^{T}v}{\partial x}$. If $v$ doesn't depend on $x$, we have
$$\frac{\partial x^{T}v}{\partial x}
= \frac{\partial x^{T}}{\partial x}v$$
But $x^{T}v = v^{T}x$, hence
$$\frac{\partial x^{T}v}{\partial x}
= \frac{\partial v^{T}x}{\partial x}
= v^{T}\frac{\partial x}{\partial x} = v^{T}$$
Thus, we conclude
$$\frac{\partial x^{T}}{\partial x}v = v^{T}$$
Similarly, we can check that
$$v^{T}\frac{\partial x}{\partial x^{T}} = v$$
A: Background info: Recall that if $f:\mathbb R^n \to \mathbb R^m$ is differentiable at $x_0 \in \mathbb R^n$, then $f'(x_0)$ is an $m \times n$ matrix which satisfies
$$
f(x) \approx f(x_0) + f'(x_0)(x - x_0).
$$
The approximation is good when $x$ is close to $x_0$.

Derivation of product rule:
Suppose that $f:\mathbb R^n \to \mathbb R^m$ and $g:\mathbb R^n \to \mathbb R^m$ are differentiable at a point $x_0 \in \mathbb R^n$, and that $h:\mathbb R^n \to \mathbb R$ is defined by 
$$h(x) = \langle f(x), g(x) \rangle
$$
for all $x \in \mathbb R^n$. Let's quickly derive a product rule to compute $h'(x_0)$. If $x$ is close to $x_0$, then
\begin{align}
h(x) &= \langle f(x), g(x) \rangle \\
&\approx \langle f(x_0) + f'(x_0)(x - x_0), g(x_0) + g'(x_0)(x - x_0) \rangle \\
&\approx \langle f(x_0), g(x_0) \rangle + \langle f'(x_0)(x - x_0), g(x_0) \rangle + \langle f(x_0), g'(x_0)(x - x_0) \rangle\\
&= h(x_0) + (g(x_0)^T f'(x_0) + f(x_0)^T g'(x_0))(x - x_0).
\end{align} 
Comparing this with the equation
$$
h(x) \approx h(x_0) + h'(x_0)(x - x_0),
$$
we see that
$$
\tag{1}
\bbox[yellow,5px]
{
h'(x_0) = g(x_0)^T f'(x_0) + f(x_0)^T g'(x_0).
}
$$
This is our product rule.

Solution:
Now let's consider the special case that $f(x) = x$ and $g(x) = Ax$, where $A$ is a real $n \times n$ matrix, so that
$h(x) = \langle x, Ax \rangle$. The derivatives of $f$ and $g$ are given by
$$
f'(x_0) = I, \qquad g'(x_0) = A.
$$
(Here $I$ is the $n \times n$ identity matrix.)
Using equation (1), we see that
\begin{align}
h'(x_0) &= (A x_0)^T I + x^T A \\
&= x_0^T(A^T + A).
\end{align}
This is what we wanted to show.
