Computing the gradient of scalar wrt a vector Let 
$$
\alpha = x^TAx \enspace x\in \mathbf{R}^{nx1}, A \in \mathbf{R}^{nxn}
$$
How do I compute the derivative $\Large \frac{\partial\alpha}{\partial x}$
without using the coordinate way i.e, writing A in terms of $\large A = (a_{ij})$
My attempt is as follows (using the product rule):
$$
\begin{align}
\frac{\partial\alpha}{\partial x} = &\frac{\partial(x^TAx)}{\partial x}\\
=&\frac{\partial(x^TA)x}{\partial x} + x^TA\frac{\partial x}{\partial x}\\
=&\frac{\partial(A^Tx)^Tx}{\partial x} + x^TAI\\
=&\left(\frac{\partial A^Tx}{\partial x}\right)^Tx+ x^TA\\
=&Ax+x^TA
\end{align}
$$
This is different from the answer that I find in wikipedia which says is to be: 
$$
\frac{\partial\alpha}{\partial x} = x^TA+x^TA^T
$$
Where am I going wrong, please present your answer in the product rule form.
 A: WE can write $\frac{\partial \alpha}{\partial x}$ in form of a row vector as:
$$
\frac{\partial \alpha}{\partial x}=x^T(A+A^T)
$$
as you find in Wikipedia, or as a column vector 
$$
\frac{\partial \alpha}{\partial x}=(A+A^T)x
$$
as you can see here at pag 11.
In your result you have the addition of a row with a column vector that is not possibile. Changing one of the two you find one of the two forms.
A: The product rule is not valid for the gradients of matrix/vector expressions.
For example
$$\frac{\partial(a^Tb)}{\partial x} \ne \Bigg(\frac{\partial a^T}{\partial x}\Bigg)b + a^T\Bigg(\frac{\partial b}{\partial x}\Bigg)$$
However, the product rule is valid for the differentials of such expressions
$$d(a^Tb) = (da^T)b + a^T(db)$$
Note that the gradient of a vector is a matrix, which behaves differently than a vector. 
But the differential of a vector is just another vector, and behaves accordingly.  
So let's approach your problem using differentials.
$$\eqalign{
 \alpha &= x^TAx \cr
d\alpha
 &= dx^T\,Ax + x^TA\,dx \cr
 &= x^TA^T\,dx + x^TA\,dx \cr
\frac{\partial \alpha}{\partial x} &= x^TA^T + x^TA \cr\cr
}$$
