Derivative of Scalarproduct? I've got this equation:
$$f(x)=\frac{1}{2}||Ax-b||^2_2 +c^Tx$$ and know that $$f'(x)=A^T(Ax-b)+c$$  but I really don't know how they derived it - maybe someone could explain it; please ? 
Thank you very much 
 A: Write down the whole in coordinates:
$$f(x)=\frac{1}{2}\sum_{i=1}^n(\sum_{j=1}^na_{ij}x_j-b_i)^2+\sum_{j=1}^nc_jx_j$$
Using that $\partial_kx_j=\delta_{jk}$ (kronecker delta), the $k$'th partial derivative is
$$\partial_kf(x)=\frac{1}{2}\sum_{i=1}^n2a_{ik}(\sum_{j=1}^na_{ij}x_j-b_i)+c_k=\sum_{i=1}^na_{ik}(\sum_{j=1}^na_{ij}x_j-b_i)+c_k\\ =\sum_{i=1}^n \sum_{j=1}^n a_{ki}^\top a_{ij}x_j-\sum_{i=1}^n a_{ki}^\top b_i+c_k=(A^\top Ax-A^\top b+c)_k$$
as claimed, where I wrote $a_{ki}^\top=a_{ik}$ where $a_{ki}^\top$ are the entries of $A^\top$.
Of course you have to understand the derivative as the differential/gradient, i.e: $$f'(x)=Df(x)=(\partial_1f,\dots,\partial_n f)^\top(x)=\nabla f(x)$$
A: Derivative with respect to a vector we will need to decide how to store the partial derivatives. If you look at the $c$ term which comes from differentiating $c^Tx$ they have clearly decided to store in column vectors in lexicographic order.
Next the term $\frac{1}{2}\|Ax-b\|_2^2 = \frac{1}{2} (Ax-b)^T(Ax-b)$
If you mind the rules of differentiation, maybe you can take it from here.
A: $\newcommand\transp{{}^{\mathrm t\mkern-2mu}}$ This comes from the derivative of the dot product:
$$ |\mkern-1.8mu| Ax-b\rVert^2=\transp(Ax-b)\cdot(Ax-b)=(\transp x\transp A-\transp b)\cdot(Ax-B)$$
so the derivative is
\begin{align*}
&(\transp x\,\transp A-\transp b)'\cdot(Ax-b)+)+(\transp x\transp A-\transp b)\cdot(Ax-b)'\newline
{}={}& \transp A\cdot (Ax-b)+(\transp x\transp A-\transp b)\cdot A\newline
{}={}& \transp A\cdot (Ax-b)+\transp A(Ax-b)=2\,\transp A\cdot (Ax-b)
\end{align*}
