derivative of quadratic function without transposes I'm trying to solve an equation of the following form:
$$ \frac{\partial}{\partial X} A'XA'X $$
where $X$ and $A$ are both equal-length column vectors (and so that $A'XA'A$ is scalar). From looking online I've seen derivatives of forms like $X'X$, but I'm unsure how it would work here when neither $X$ is transposed. My guess was
$$ 2A'AX $$
just to ensure that the derivative is also scalar, but I don't have confidence in that.
 A: Two items; given a square symmetric matrix $B$ and a column vector $X$ that fits, the gradient of $X'BX,$ written as a column, is indeed $2BX.$
Next, the transpose of a one by one matrix is always itself; what else could it be? So $$  A'X = (A'X)' = X'A'' = X' A.  $$ As a result, your expression is
$$ A'X A' X = (A'X) A' X = X' A A' X = X' (A A') X.   $$
Taking symmetric square $B = A A'$ we get $X' B X.$
Your final expression needs to be $$  2 A A' X.  $$ Yours makes no sense; there is no such thing as $AX.$ If you meant $2(A'A)X$ it is still wrong.
I suggest you write this out in full detail for  dimension 1 and dimension 2. For dimension 2, give $A$ constant entries $a,b$ and give $X$ entries $x,y.$ As a column vector, what is the gradient of $(ax+by)^2 = a^2 x^2 + 2 a b x y + b^2 y^2?$
Using dot product, the gradient, as a row, is 
$$ 2 (A \cdot X) A',  $$
as a column
$$ 2 (A \cdot X) A.  $$
A: Define a scalar
$$b=A'X$$
Then you get
$$\frac{\partial }{\partial X}b^2=2b\frac{\partial b}{\partial X}$$
With
$$\frac{\partial b}{\partial X}=A$$
you finally get for the derivative
$$2bA=2(A'X)A$$
A: As $(A'X)$ is a scalar, we can derive the $i$th component of that derivative as
$$\left(\frac{\partial}{\partial X} A'XA'X\right)_i = \partial_i (A'XA'X) = \partial_i (A'X)^2 \overset{(1)}{=} 2(A'X) \partial_i A'X=2(A'X) \underbrace{\partial_i \sum_{k}A_k^* X_k}_{\displaystyle{=A_i^*}}$$ where we used the chain rule for (1) and hence 
$$\frac{\partial}{\partial X} A'XA'X=2(A'X)A'.$$
Note that this differs from the other answers, which give you a column vector $2(A'X)A$, hopefully I'm right :D
