What is the derivative of this? I have a function of the following form:
$J = \|W^TW-I\|_F^2$ 
Where, $W$ is a matrix and $F$ is the Frobenius Norm.
How can I find the derivative of $\frac{\partial J}{\partial W}$ ?
 A: You can use the product rule for matrix differentiation, 
$$
\frac{\partial\mathrm{trace}(f(X)G(X))}{\partial X} = \left. \frac{\partial\mathrm{trace}(f(X)G(Y))}{\partial X}+\frac{\partial\mathrm{trace}(f(Y)G(X))}{\partial X}\right|_{Y=X}
$$
along with the simpler identity
$$
\frac{\partial\mathrm{trace}(AX)}{\partial X}=A^\top.
$$
For your problem this would go as follows.  First write the problem in this form:
\begin{eqnarray*}
J &=& \|W^\top W-I\|_F^2 \\
&=& \mathrm{trace}((W^\top W-I)^\top(W^\top W-I))\\
&=& \mathrm{trace}(W^\top WW^\top W - 2W^\top W + I)
\end{eqnarray*}
The derivative can now be computed much as for a scalar function.  First we split it up into two parts
\begin{eqnarray*}
J'(W) &=& \frac{\partial}{\partial W}\mathrm{trace}(W^\top WW^\top W - 2W^\top W + I)\\
&=& \frac{\partial}{\partial W}\mathrm{trace}(W^\top WW^\top W) - 2\frac{\partial}{\partial W}\mathrm{trace}(W^\top W)\\
&=& J_1'(W) -2 J_2'(W)
\end{eqnarray*}
Starting with the simplest term we have 
\begin{eqnarray*}
J_2'(W) &=& \frac{\partial}{\partial W}\mathrm{trace}(W^\top W)\\
&=& \left. \frac{\partial}{\partial W}\mathrm{trace}(W^\top Y + Y^\top W)\right|_{Y=W}\\
&=& \left.Y + (Y^\top)^\top\right|_{Y=W}\\
&=& 2W
\end{eqnarray*}
And the quartic term:
\begin{eqnarray*}
J_1'(W) &=& \frac{\partial}{\partial W}\mathrm{trace}(W^\top WW^\top W)\\
&=& \left. \frac{\partial}{\partial W}\mathrm{trace}(W^\top W Y^\top Y + Y^\top Y W^\top W)\right|_{Y=W}\\
&=& 2\left. \frac{\partial}{\partial W}\mathrm{trace}(W^\top W Y^\top Y)\right|_{Y=W}\\
&=& 2\left. \frac{\partial}{\partial W}\mathrm{trace}(W^\top Y Y^\top Y + Y^\top W Y^\top Y )\right|_{Y=W}\\
&=& \left. 2 Y Y^\top Y  + 2(Y^\top Y Y^\top ) ^\top\right|_{Y=W}\\
&=& 4 W W^\top W
\end{eqnarray*}
So the derivative of $J$ should be
$$ J'(W) =  4 W W^\top W - 4 W = 4 W(W^\top W-I). $$
A: Assuming the norm comes from real inner-product. The derivative of the norm can be found as
$$\left.\frac{d}{dt}\right|_{0}\|(W+tH)^T(W+tH)-I\|^{2}\\
=\left.\frac{d}{dt}\right|_{0}\langle (W+tH)^T(W+tH)-I,(W+tH)^T(W+tH)-I\rangle\\
=2\langle W^TW-I,H^TW+W^TH\rangle$$
We now use the definition of adjoint corresponding to a simple matrix transpose for real matrices, $\langle Ax,y\rangle=\langle x,A^Ty\rangle$ and similarly for the operators acting from the right; as well as again the symmetry of the real inner product, we find that the two H-linear terms are in fact the same:
$$
\langle W^TW-I,W^TH\rangle=\langle W(W^TW-I),H\rangle
$$
$$
\langle W^TW-I,H^TW\rangle=\langle H(W^TW-I),W\rangle\\=\langle H,W(W^TW-I)^T\rangle\\
=\langle H,W(W^TW-I)\rangle\\
=\langle W(W^TW-I),H\rangle
$$
Summing up, the derivative is the linear map
$$
4\langle W(W^TW-I),H\rangle
$$
Then, and only because we have the inner product, we can write the gradient vector: $$\frac{\partial J}{\partial W}=4W(W^TW-I)$$
