General question about matrix calculus with specific example (with attempted answer) I'm struggling to find the right way to approach matrix calculus problems generally. As an example of a problem that is bothering me, I would like to calculate the derivative of $||Ax||$ (Euclidean vector norm) with respect to the matrix $A$. How can I discover this via first principles? The natural thing seems to be to consider $||(A+H)x||-||Ax||$ as $||H||$ goes to zero but I don't see how to get something tangible from it. 
Addendum: This question is getting little attention. I am really looking for a general approach for solving these sorts of matrix calculus problems. In particular, finding the derivative with respect to a matrix of certain vector quantities. This comes up all the time in convex optimization algorithms like gradient descent and so on.\
Further: If we look at the derivative of $||Ax||^2$ with respect to $A$ we see that this expression can be written as trace$(Axx^TA^T$), so the derivative with respect to $A$ is $2xx^TU^T$.
Edit: I don't know if this is the Frechet derivative per se, but I guess we can just notice that $||Ax||^p=(||Ax||^2)^{\frac{p}{2}}$, so by the power rule we get that the derivative of this is $p\cdot\frac{xx^T U^T}{||Ax||^{p/2 - 1}}$. Is this correct??
 A: Here is something which might help answer your question. Let $A \in \mathcal{M}_{n}(\mathbb{R})$ and $x \in \mathbb{R}^{n}$, $x \notin \mathrm{ker}(A)$ (because if $x \in \mathrm{ker}(A)$, then $Ax=0$). If I follow your first post, to find the differential of $A \in \mathcal{M}_{n}(\mathbb{R}) \, \longmapsto \, \Vert Ax \Vert$, you can consider : $\Vert (A+H)x \Vert - \Vert Ax \Vert$ for some matrix $H$.
First notice that :
$$ \Vert Ax + Hx \Vert - \Vert Ax \Vert = \frac{\Vert Ax + Hx \Vert^{2} - \Vert Ax \Vert^{2}}{\Vert Ax+Hx \Vert + \Vert Ax \Vert} $$
Where $\Vert Ax+Hx \Vert^{2} - \Vert Ax \Vert^{2} = 2 \left\langle Ax,Hx \right\rangle + \Vert Hx \Vert^{2}$ (where $\left\langle \cdot,\cdot \right\rangle$ denotes the usual dot product on $\mathbb{R}^{n}$). So,
$$ \Vert Ax + Hx \Vert - \Vert Ax \Vert = \frac{2\left\langle Ax,Hx \right\rangle + \Vert Hx \Vert^{2}}{\Vert Ax + Hx \Vert + \Vert Ax \Vert}$$
The RHS can be written :
$$ \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle + \varepsilon_{1}(H) + \varepsilon_{2}(H)$$
where both $\varepsilon_{1}$ and $\varepsilon_{2}$ depend on $A$ and $x$. In fact, we have :
$$ \varepsilon_{1}(H) = - \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle \frac{\Vert Ax + Hx \Vert - \Vert Ax \Vert}{\Vert Ax + Hx \Vert + \Vert Ax \Vert}$$
and
$$ \varepsilon_{2}(H) = \frac{\Vert Hx \Vert^{2}}{\Vert Ax + Hx \Vert + \Vert Ax \Vert}$$
To conclude, one has to check that $\varepsilon_{1}(H) = o(\Vert H \Vert)$ and $\varepsilon_{2}(H) = o(\Vert H \Vert)$. By definition, $\Vert Hx \Vert \leq \Vert H \Vert \Vert x \Vert$. So, for $\varepsilon_{2}$, we have :
$$ \varepsilon_{2}(H) \leq \frac{\Vert H \Vert^{2} \Vert x \Vert^{2}}{\Vert Ax + Hx \Vert + \Vert Ax \Vert}$$
Which proves that $\lim \limits_{\Vert H \Vert \rightarrow 0} \frac{\varepsilon_{2}(H)}{\Vert H \Vert} = 0$. For $\varepsilon_{1}$,
$$ \vert \varepsilon_{1}(H) \vert \leq \Bigg\vert \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle \Bigg\vert \frac{\Big\vert \Vert Ax + Hx \Vert - \Vert Ax \Vert \Big\vert}{\Vert Ax + Hx \Vert + \Vert Ax \Vert} $$
And :
$$ \Bigg\vert \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle \Bigg\vert \leq \Vert Hx \Vert $$
by Cauchy-Schwarz inequality and $\Big\vert \Vert Ax + Hx \Vert - \Vert Ax \Vert \Big\vert \leq \Vert Hx \Vert$ by triangular inequality. It gives :
$$ \vert \varepsilon_{1}(H) \vert \leq \frac{\Vert Hx \Vert^{2}}{\Vert Ax+Hx \Vert + \Vert Ax \Vert}$$
Eventually, $\lim \limits_{\Vert H \Vert} \frac{\varepsilon_{1}(H)}{\Vert H \Vert} = 0$. In conclusion :

$$ \Vert (A+H)x \Vert - \Vert Ax \Vert = \Vert Ax + Hx \Vert - \Vert Ax \Vert = \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle + \varepsilon(H) $$
where $\varepsilon(H) = o(\Vert H \Vert)$.

Since $H \, \longmapsto \, \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle$ is linear, we can say that :

$$ \mathrm{D}_{A} \Vert Ax \Vert \cdot H = \left\langle \frac{Ax}{\Vert Ax \Vert},Hx \right\rangle = \frac{{}^t x {}^t A H x }{\Vert Ax \Vert} $$

(where ${}^t x$ denotes the transpose of $x$).
A: In the following the vector $x\in{\mathbb R}^n$ is fixed. We begin by considering the function
$$g:\quad M_{n\times n}\to{\mathbb R}, \qquad A\mapsto g(A):=|Ax|^2\ .$$
One has
$$g(A+H)=\langle(A+H)x,(A+H)x\rangle=|Ax|^2+2\langle Ax,Hx\rangle+|Hx|^2\ .$$
As $$|Hx|^2\leq \|H\|^2 |x|^2=o\bigl(\|H\|\bigr)\qquad(H\to0)$$
we have
$$g(A+H)-g(A)=2\langle Ax,Hx\rangle+o\bigl(\|H\|\bigr)\qquad(H\to0)\ .$$
This proves
$$dg(A).H=2\langle Ax,Hx\rangle\ .$$
Now we are really interested in the function $f:={\rm sqrt}\circ g$. Since ${\rm sqrt}:\ t\mapsto \sqrt{t}$ is differentiable only for $t>0$ we have to assume $Ax\ne0$ in the sequel. Using the chain rule and $${\rm sqrt}'(t)={1\over 2\sqrt{t}},\quad{\rm i.e.,}\quad d{\rm sqrt}(t).T={1\over 2\sqrt{t}}\>T,$$we obtain
$$df(A).H=d{\rm sqrt}\bigl(g(A)\bigr).\bigl(dg(A).H\bigr)={\langle Ax,Hx\rangle\over |Ax|}\ .$$
A: Everything can be settled down with the chain rule. For definiteness let us write 
\begin{align}
N(y)&=\lVert y \rVert_2, \qquad y\in \mathbb{R}^n, \\
M_x(A)&=Ax, \qquad A\in \mathbb{R}^{n\times n}
\end{align}
We have 
$$N\colon \mathbb{R}^n\to \mathbb{R},\quad M_x\colon \mathbb{R}^{n\times n}\to \mathbb{R}^n,$$
so ($\circ$ denotes composition):
$$N\circ M_x\colon \mathbb{R}^{n\times n}\to \mathbb{R}.$$
We want to compute its derivative, which is to be taken in Fréchet sense. Therefore, for fixed $A\in \mathbb{R}^{n\times n}$, 
$$d_A(N\circ M_x)\colon \mathbb{R}^{n\times n} \to \mathbb{R}$$
is a linear map, or linear functional as its range is $\mathbb{R}$. The chain rule tells us that 
$$\tag{1}d_A(N\circ M_x)=d_{M_x(A)} N\circ d_{A}M_x.$$
Now $M_x$ is already a linear operator, so $d_AM_x$ is just $M_x$ itself. On the other hand, $d_y N$ exists only for $y\ne 0$, in which case 
$$\begin{split}
d_y N(h)&=\left.\frac{d }{d\epsilon}\right|_{\epsilon=0} N(y+\epsilon h) \\
&=\left.\frac{d }{d\epsilon}\right|_{\epsilon=0}\sqrt{(y_1+\epsilon h_1)^2+\dots+(y_n+\epsilon h_n)^2}\\
&=\frac{y\cdot h}{\lVert y \rVert_2}.
\end{split}$$
Combining this with (1) we get 
$$
\begin{split}
d_A(N\circ M_x)(B)&= d_{M_x(A)}N(d_A M_x(B)) \\
 &= d_{M_x(A)}N (M_x(B))\\ 
&=d_{M_x(A)}N(Bx)=\frac{Ax\cdot Bx}{\lVert Ax\rVert_2}.\end{split}
$$
This is a possible way to express the result, which admittedly is more theoretical-friendly than computational-friendly. 
Another possibility is to consider partial derivatives:
$$\tag{2}\frac{\partial}{\partial A_{ij}} \lVert Ax\rVert_2=\frac{\partial}{\partial A_{ij}}\sqrt{\sum_{hk} (A_{hk}x_k)^2}=\frac{A_{ij} x_j^2}{\lVert Ax \rVert_2}.$$ 
The connection between those two results is the following. Any linear functional on $\mathbb{R}^{n\times n}$ can be identified with an element of $\mathbb{R}^{n \times n}$, we may simplify things a bit by identifying $d_A(N\circ M_x)$ with a matrix which we call gradient:
$$d_A(N\circ M_x)\equiv\nabla (N\circ M_x)(A).$$
 The identification is performed by means of the following equation:
$$d_A(N\circ M_x)(B)=\langle \nabla (N\circ M_x)(A), B\rangle, $$ 
where the scalar product $\langle, \rangle$ between two matrices is given by 
$$\langle X, Y\rangle = \sum_{hk} X_{hk} Y_{hk}.$$
Now it turns out that the components of the matrix $\nabla(N\circ M_x)(A)$ are precisely the partial derivatives we computed in (2). This is a general fact. Therefore, in actual computations, it is often simpler and quicker to just compute partial derivatives.
A: One approach is to use the equation 
\begin{equation}
f(A + \Delta A) \approx f(A) + \langle \nabla f(A), \Delta A \rangle.
\end{equation}
For example suppose $f(A) = \|Ax\|^2$.  Then
\begin{align}
f(A + \Delta A) &= \|Ax + \Delta A x \|^2 \\
&= \|Ax\|^2 + 2 (Ax)^T \Delta A x  + \| \Delta A x \|^2 \\
&\approx \|Ax\|^2 + 2  (Ax)^T \Delta A x \\
&= f(A) + 2\, \text{Tr}( (Ax)^T \Delta A x ) \\
&= f(A) + 2\,\text{Tr}(\Delta A x (Ax)^T) \\
&= f(A) + 2\,\text{Tr}(\Delta A x x^T A^T) \\
&= f(A) + \langle 2Ax x^T , \Delta A \rangle.
\end{align}
This shows that $\nabla f(A) = 2Axx^T$.
I used the fact that $\text{Tr}(AB) = \text{Tr}(BA)$ when both products $AB$ and $BA$ are defined.
A: Use a colon denote the trace/Frobenius product, i.e. $\, A:B = {\rm Tr}(A^TB)$
The properties of the trace allow its terms to be rearranged in a variety of ways, e.g.
$$\eqalign{
A:B &= B:A \\
A:B &= A^T:B^T \\
A:BC &= AC^T:B = B^TA:C \\
}$$
Write the norm in terms of the Frobenius product.
Then calculating its differential and gradient is quite straightforward.
$$\eqalign{
\lambda &= \|Ax\| \\
\lambda^2 &= \|Ax\|^2 = Ax:Ax \\
2\lambda\,d\lambda &= 2Ax:dA\,x \\
\lambda\,d\lambda &= Axx^T:dA \\
\frac{\partial\lambda}{\partial A} &= \frac{Axx^T}{\lambda} \\
}$$
