Matrix Differentiation :: Resource Needed I was reading a manuscript and encountered something I had never seen before.
\begin{align}
\phi(X)&=\frac{1}{2}\|XY-K\|_F^2\\
\phi'(X)&=(XY-K)Y^\dagger
\end{align}
where $X\in\mathbb{R}^{m\times p}$, $Y\in\mathbb{R}^{p\times n}$ and $K\in\mathbb{R}^{m\times n}$
$\|\bullet \|_F$ implies Frobenius norm and $Y^\dagger$ is the Moore-Penrose Pseudo inverse.
Firstly, the manuscript is laden with errors so I am not sure if this is true but it does have an intuitive appeal. Secondly, I have studied some matrix calculus but never encountered something like this. Can someone give me a reference to read about such differentiations?
I also have access to the wonderful cookbook according to which (Formula 108), the Moore-Penrose should be replaced by Transpose but I can't seem to find a proof. 
I was not sure if I should put this on Math.SE or here but I think this is a little too advanced for Math.SE
 A: The formula you quote is inaccurate in two aspects: Firstly, it should read $Y^\top$ (transpose) not $Y^\dagger$ (as you suggested yourself), and secondly, this is not the formula for the derivative of $\phi$ but for its gradient with respect to the Frobenius inner product (to be more precise, with respect to the Riemannian metric induced on the matrix manifold $M=\mathbb{R}^{m\times p}$ by the Frobenius inner product). The latter is a common mistake in the engineering literature (I work in an engineering faculty).
Here is a suggestion how you can verify the formula yourself using calculus on manifolds. I am providing some level of detail here because I know how many semi-accurate sources on this type of calculus are out there.
Start by rewriting $\phi(X)=\frac{1}{2}\text{trace}((XY-K)^\top(XY-K))$.
For a differentiable map $f\colon M\rightarrow P$, where $P$ is some arbitrary manifold, denote the derivative at a point $X\in M$ in direction $H\in T_X M\equiv\mathbb{R}^{m\times p}$ by $Df(X)\cdot H$. Then the product rule and the chain rule imply 
$$D\phi(X)\cdot H=\frac{1}{2}\text{trace}((HY)^\top(XY-K)+(XY-K)^\top(HY)).$$
Here I have used the elementary facts that the derivative of a constant map is zero and that differentiation commutes with (can be pulled through) linear maps. Note that $\text{trace}$ is linear in its matrix argument and so are the map that multiplies a matrix with a constant matrix and the matrix transpose. This way you work from the outside all the way to the $X$ inside the formula (twice, since you have a product of two terms each containing $X$) and then you use the fact that the derivative of the identity map $\text{id}\colon X\mapsto X$ is the identity map, so $D\,\text{id}(X)\cdot H=H$.
Now simplify using properties of the trace:
$$D\phi(X)\cdot H=\text{trace}(H^\top(XY-K)Y^\top)=\langle H,(XY-K)Y^\top\rangle,$$
where I have denoted the Frobenius inner product by $\langle.,.\rangle$. Now recall the relationship between a gradient and a directional derivative to see that
$$\text{grad}\ \phi(X)=(XY-K)Y^\top.$$
The same ideas work for more general matrix manifolds but then you might need to use projectors to the tangent space in the calculus, see e.g. P.-A. Absil, R. Mahony, and R. Sepulchre, "Optimization on matrix manifolds", Princeton Uiversity Press, 2008.
