Matrix differentiation with respect to matrix I have an $n \times r$ matrix $A$ and an $r \times m$ matrix $B$. 
What will the derivative of $ABB^T$ with respect to $A$ be? Is it simply $BB^{T}$? 
 A: Consider the matrix-valued function
$$\mathrm F (\mathrm X) := \mathrm X \mathrm C$$
The $(i,j)$-th entry of the output is
$$f_{ij} (\mathrm X) := \mathrm e_i^\top \mathrm F (\mathrm X) \,\mathrm e_j = \mathrm e_i^\top \mathrm X \mathrm C \,\mathrm e_j = \mathrm e_i^\top \mathrm X \,\mathrm c_j = \mbox{tr} \left( \mathrm c_j\mathrm e_i^\top \mathrm X \right) = \langle \mathrm e_i\mathrm c_j^\top, \mathrm X \rangle$$
where $\mathrm c_j$ is the $j$-th column of $\mathrm C$ and $\langle \cdot, \cdot \rangle$ denotes the Frobenius inner product. Hence, the gradient of $f_{ij}$ with respect to $\mathrm X$ is
$$\nabla f_{ij} (\mathrm X) = \mathrm e_i\mathrm c_j^\top$$
A: This answer provides a different way of writing the results than that provided by Rodrigo.  I have every belief his analysis to be correct, but I personally find this particular method a little more intuitive and general without having to introduce tensor concepts.
Consider the operation of restructuring a matrix as a long vector by concatenating successive columns.  For a given matrix $W$, this operator is often given the notation $vec\left( W \right)$ (see this Wikipedia page).  Then, there exists a useful property that for any three matrices $X$, $Y$ and $Z$
$$vec\left( {XYZ} \right) = \left( {{Z^T} \otimes X} \right)vec\left( Y \right)$$
where $\otimes$  is the Kronecker product (see this Wikipedia page).
Now the derivative of an $n\,x\,m$ matrix $W$ WRT itself is
$$\frac{{\partial W}}{{\partial W}} = \frac{{\partial \,vec\left( W \right)}}{{\partial \,vec\left( W \right)}} = {I_{\left[ {nm} \right]}}$$
Then, for $X = {I_{\left[ n \right]}}$, $Y = A$, and $Z = B{B^T}$
$$vec\left( {AB{B^T}} \right) = \left( {B{B^T} \otimes {I_{\left[ n \right]}}} \right)vec\left( A \right)\quad  \Rightarrow \quad \frac{{\partial \left( {AB{B^T}} \right)}}{{\partial A}} = B{B^T} \otimes {I_{\left[ n \right]}}$$
