Questions on "painless conjugate gradient": take gradient of a quadratic form I am reading this paper:
http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf
I have difficulties on the derivation of equation (6) on page 4.
It is to take gradient of a quadratic form.
I searched around and found this: How to take the gradient of the quadratic form?
I can understand most of the answer in above link, but:


*

*Why the $y$ in the second part of chain rule needs to be transposed?

*In neither original paper or above Q/A it tells me how to take derivative of a vector valued function($R^n \rightarrow R^n$). I think that was used implicitly in the derivation of
$\dfrac{\partial (x^TA^T)}{\partial x}$. And that may be not rigorous to apply 
$$\dfrac{\partial (b^Tx)}{\partial x} = \dfrac{\partial (x^Tb)}{\partial x} = b$$
directly on $\dfrac{\partial (x^TA^T)}{\partial x}$ to get $A^T$.

 A: Well, never mind. Your matrix $A$ is not necessarily symmetric, which is annoying. But then you take $$ f(x) = \frac{1}{2} x^T A x - b^T x + c, $$ where $B$ is a constant column vector, and $x$ is the column vector with entries $x_j,$ the $n$ coordinate functions. For your purposes, perhaps the choices are $n=1,2,3.$
So $$ f(x) = \frac{1}{2} \left( \sum_{i=1}^n  \sum_{j=1}^n a_{ij} x_i x_j \right) - \left( \sum_{k=1}^n  b_k x_k \right) + c.  $$
I want to fix an index $m$ and find $\frac{\partial f}{\partial x_m}.$  The terms in $f$ that involve $x_m$ are exactly
$$  \frac{1}{2} \left( a_{mm} x_m^2 + \sum_{i \neq m}^n   a_{im} x_i x_m + \sum_{j \neq m}^n   a_{mj} x_m x_j \right) -   b_m x_m .  $$
As a result, 
$$ \frac{\partial f}{\partial x_m} =  a_{mm} x_m + \frac{1}{2}  \sum_{i \neq m}^n   a_{im} x_i  + \frac{1}{2} \sum_{j \neq m}^n   a_{mj}  x_j  -   b_m $$
Next, separate $$  a_{mm} x_m  =  \frac{1}{2} a_{mm} x_m + \frac{1}{2}a_{mm} x_m $$ and stick the two pieces back into the sums, arriving at
$$ \frac{\partial f}{\partial x_m} =   \frac{1}{2}  \sum_{i =1}^n   a_{im} x_i  +  \frac{1}{2} \sum_{j = 1}^n   a_{mj}  x_j  -   b_m $$
The traditional thing is to replace $i$ by $j$ in the first sum, written using the matrix $A^T;$ so
$$ \frac{\partial f}{\partial x_m} =   \frac{1}{2}  \sum_{j =1}^n   a_{mj}^T x_j  +  \frac{1}{2} \sum_{j = 1}^n   a_{mj}  x_j  -   b_m $$
So, we get the gradient of $f,$ written as a column vector, as
$$  \nabla f = \frac{1}{2} \left( A^T + A \right) x - b. $$
That's about it. The new matrix $$  \frac{1}{2} \left( A^T + A \right)  $$ is called the symmetric part of $A,$ and it is equal to $A$ itself if $A$ is already symmetric. 
