need help with matrix calculus I am trying to find $\frac{\partial  (x'Ax)}{\partial x}$ where x is a vector (2 x 1 vector) and A is a matrix (say 2x2 dimensions).
When I looked up in http://www.matrixcalculus.org/ I found the answer to be $(A.x)' + x'.A$ where $'$ stands for transpose.
I tried to solve this using $\frac{\partial (CB)}{\partial x} = \frac{\partial{C}}{\partial x}B + C\frac{\partial B}{\partial x}$ where $C = x'A$ and $B = x$. With this in perspective, I am getting the derivative as $A'x + x'A$. 
Clearly with $x$ being a column vector and $A$ being a square matrix, my answer is wrong since individual terms ($A'x$ and $x'A$)have different shapes. 
Where am I getting the calculations wrong. Any help ? 
 A: The rule 
$\frac{\partial \mathbf C\mathbf B}{\partial x} = \frac{\partial{\mathbf C}}{\partial x}\mathbf B + \mathbf C\frac{\partial \mathbf B}{\partial x}$
is for differentiation by a scalar $x$ where $\mathbf B$ and $\mathbf C$ are matrices. It is not a rule for differentiation by a vector.
With $\mathbf C = \mathbf x^\top \mathbf A$ and $\mathbf B = \mathbf x,$ you have a row vector $\mathbf C$ and a column vector $\mathbf B,$
so you can apply the rule
$$
\frac{\partial \mathbf u^\top\mathbf v}{\partial \mathbf x}
 = \mathbf u^\top \frac{\partial{\mathbf v}}{\partial \mathbf x}
 + \mathbf v^\top \frac{\partial{\mathbf u}}{\partial \mathbf x}
$$
with $\mathbf u^\top = \mathbf C$ and $\mathbf v = \mathbf B,$ so
\begin{align}
\frac{\partial \mathbf C\mathbf B}{\partial \mathbf x}
 &= \mathbf C \frac{\partial{\mathbf B}}{\partial \mathbf x}
 + \mathbf B^\top \frac{\partial{\mathbf C^\top}}{\partial \mathbf x}\\
 &= \mathbf x^\top \mathbf A \frac{\partial{\mathbf x}}{\partial \mathbf x}
 + \mathbf x^\top \frac{\partial{\mathbf A^\top\mathbf x}}{\partial \mathbf x}\\
&= \mathbf x^\top \mathbf A + \mathbf x^\top \mathbf A^\top\\
&= \mathbf x^\top \mathbf A + (\mathbf A\mathbf x)^\top.
\end{align}
Done!

For differentiation of matrices by vectors, refer to
Derivative of a Matrix with respect to a vector.
But that seems much more than you need or want.
