Derivative of Kronecker product with chain rule? This is somewhat related to my previous question, but in a slightly different form and with different assumptions.
Say I have the following expression:
$\frac{\delta E}{\delta \mathbf{R}} = 2\Theta \mathbf{R}^T - 2\Phi$
where $\Phi, \Theta \text{ and } \mathbf{R}$ are all matrices.
Now, given the additional constraint:
$\mathbf{R} = \mathbf{B}(\mathbf{I} \otimes \mathbf{K})$
($\mathbf{I}$ is an identity matrix, $\mathbf{B}$ is also a matrix)
How does one find $\frac{\delta E}{\delta \mathbf{K}}$?
 A: Reverting to the differential expression
$$\eqalign{
dE &= 2\,(\Theta R^T-\Phi):dR \cr
   &= 2\,(\Theta R^T-\Phi):B(I\otimes dK) \cr
   &= 2\,B^T(\Theta R^T-\Phi):I\otimes dK \cr
   &= M:I\otimes dK \cr\cr
}$$
Now you need the Kronecker factorization of $M$ 
$$\eqalign{
  M &= \sum_{j=1}^r A_j\otimes Z_j \cr
}$$
where the $A_j$ matrices are shaped like $I$, and the $Z_j$ like $K$.
Then apply the Frobenius-Kronecker mixed product rule
$$\eqalign{
  (A\otimes Z):(E\otimes F) &= (A:E)\,(Z:F) \cr
}$$
to get
$$\eqalign{
 dE &= \sum_{i=j}^r A_j\otimes Z_j : I\otimes dK \cr
    &= \sum_{i=j}^r (A_j:I)\,Z_j:dK \cr
    &= \sum_{i=j}^r {\rm tr}(A_j)\,Z_j:dK \cr\cr
\frac{\partial E}{\partial K} &= \sum_{j=1}^r\,Z_j\,{\rm tr}(A_j) \cr\cr
}$$
For help with the Kronecker factorization search for articles by van Loan and Pitsianis.
A: $\def\o{{\tt1}}\def\p{{\partial}}\def\grad#1#2{\frac{\p #1}{\p #2}}\def\l{\left}\def\r{\right}$Start
with hans's differential relation and rearrange it.
Assume that $K$ is $\,(m\times n)\,$ and $\,I_p$ is $\,(p\times p)$
$$\eqalign{
dE &= M:I_p\otimes dK \\
 &= M:\sum_{k=1}^p \l(e_ke_k^T\otimes dK\r) \\
 &= M:\sum_{k=1}^p \l(e_k\otimes I_m\r)\l(I_\o\otimes dK\r)\l(e_k\otimes I_n\r)^T \\
 &= \sum_{k=1}^p \l(e_k\otimes I_m\r)^TM\l(e_k\otimes I_n\r):\l(I_\o\otimes dK\r) \\
 &= \sum_{k=1}^p \l(e_k\otimes I_m\r)^TM\l(e_k\otimes I_n\r):dK \\
}$$
where $e_k$ is the $k^{th}$ column of $I_p\,$ and $I_\o$ is the $(\o\times\o)$ identity matrix.
Now that $dK$ has been isolated on the RHS the gradient can be trivially calculated as
$$\eqalign{
\grad{E}{K} &= \sum_{k=1}^p \l(e_k\otimes I_m\r)^TM\l(e_k\otimes I_n\r) \\
}$$
