Differentiation of the euclidean norm of $\Vert Ax+b\Vert^{2}$ Let $A$ be an $m\times n$ real matrix, $x$ an $n\times 1$ vector and $b$ an $m\times 1$ vector. I want to compute
\begin{equation}
\dfrac{\partial }{\partial x} \Vert Ax+b\Vert^{2}.
\end{equation}
First, I expanded
\begin{equation}
\Vert Ax+b\Vert^{2}=(Ax+b)^{T}(Ax+b)=x^{T}A^{T}Ax+2x^{T}A^{T}b+b^{T}b
\end{equation}
then I computed
\begin{eqnarray}
\dfrac{\partial }{\partial x}(x^{T}A^{T}Ax+2x^{T}A^{T}b+b^{T}b)=A^{T}Ax+x^{T}A^{T}A+2A^{T}b
\end{eqnarray}
but I know the above is wrong since $A^{T}Ax$ and $x^{T}A^{T}A$ does not have the same dimention. Thanks for the help.
 A: Rather than expanding first, do the opposite.  Define a new vector $$y=Ax+b$$ and write the function in terms of this new variable and the Frobenius product (which I'll denote by a colon). This approach reduces the visual "clutter". You can then expand the results after finding the derivative.
With the Frobenius product, finding the gradient is easy and fool-proof 
$$\eqalign{
 f &= \|y\|^2 = y:y \cr
\cr
df &= 2\,y:dy \cr
  &= 2\,y:A\,dx \cr
  &= 2\,A^Ty:dx\cr
\cr
\frac{\partial f}{\partial x} &= 2\,A^Ty \cr
  &= 2\,A^T(Ax+b) \cr
\cr
}$$
The rules for rearranging the Frobenius product
$$\eqalign{
 A:B &= B:A \cr
 A:BC &= B^TA:C = AC^T:B\cr
}$$
can be derived from the familiar properties of the trace, since 
$$A:B={\rm tr}(A^TB)$$
A: A general way of finding the gradient $\nabla_x f(x)$ of any vector-valued function $f:\mathbb{R}^n \rightarrow \mathbb{R}$ is by using Taylor's Theorem, which can be expressed as such:
$$f(x+\delta) = f(x) + [\nabla_x f(x)]^\intercal \delta + o(||\delta||_2),$$
where $\delta\in \mathbb{R}^n$ is small.
For your example, let $f(x) := ||Ax+b||_2^2$. Applying Taylor's Theorem (above), 
we have that:
\begin{align}
f(x + \delta) &= ||A(x+\delta)+b||_2^2 \\
&= ||Ax+b + A\delta||_2^2\\
&= ||Ax + b||_2^2 + 2(Ax+b)^\intercal A\delta + ||A\delta||_2^2\\
&= f(x) + [2A^\intercal(Ax+b)]^\intercal \delta + o(||\delta||_2)\\
&= f(x) + [\nabla_x f(x)]^\intercal \delta + o(||\delta||_2).
\end{align}
Thus, the gradient $\nabla_x f(x)=2A^\intercal(Ax+b)$.
(The third line in the equations above is given by the equality
$||U+V||_2^2 = ||U||_2^2 + 2U^\intercal V + ||V||_2^2$, for any vectors $U, V \in \mathbb{R}^n$.)
