How does one evaluate the derivative of a matrix with a tensor $\frac{\partial \operatorname{Tr}[A(\mathrm{Id}\otimes w)]}{\partial w}$? I am stuck on the following:
$$\frac{\partial \operatorname{Tr}[A(\mathrm{Id}\otimes w)]}{\partial w}=\text{ ?}$$
with $A$ a $d\times d^2$ matrix, $\mathrm{Id}$ the identity matrix of $d\times d$ dimension and $w$ a $d\times 1$ vector.  
THANKS!!!
 A: Using the technique from this question, first calculate the Kronecker factorization $$A^T=Y\otimes z$$where $(Y,z)$ are shaped like $(I,w)$ respectively.
Using this factorization and the Frobenius (:) Inner Product you can rewrite the function, then find its differential and gradient 
$$\eqalign{
  f &= A^T:(I\otimes w) \cr\cr
 df &= (Y\otimes z):(I\otimes dw) \cr
    &= (I:Y)\,(z:dw) \cr
    &= {\rm tr}(Y)\,z:dw \cr
\frac{\partial f}{\partial w} &= z\,{\rm tr}(Y) \cr\cr
}$$
In most cases, you'll need a sum of Kronecker factors
$$A^T = \sum_{k=1}^r Y_k\otimes z_k$$
which changes the result slightly
$$\eqalign{
\frac{\partial f}{\partial w} &= \sum_{k=1}^r z_k\,{\rm tr}(Y_k) \cr\cr
}$$
A: The Singular Value Decomposition 
$$\eqalign{
A &= \sum_{k=1}^{r} \sigma_k u_k v_k^T \cr
\sigma_k&\in{\mathbb R},\,\,\,u_k\in{\mathbb R}^{d\times 1},\,\,\,v_k^T\in{\mathbb R}^{1\times d^2}
}$$
can be used to find the gradient of this function.
$$\eqalign{
\phi &= I:(I\otimes w)\,A \cr
d\phi &= I:(I\otimes dw)\,A \cr
 &= \sum_kI:(I\otimes dw)\, \sigma_k u_k v_k^T \cr
 &= \sum_k\sigma_k v_k:(I\otimes dw)\,u_k \cr
 &= \sum_k\sigma_k v_k:{\rm vec}(dw\,u_k^T) \cr
 &= \sum_kM_k:dw\,u_k^T \cr
 &= \sum_kM_ku_k:dw \cr
\frac{\partial\phi}{\partial w} &= \sum_kM_ku_k \cr
}$$
NB:  The $\{M_k\}$ are $(d\times d)$ matrices such that 
$\,{\rm vec}(M_k) = \sigma_k v_k$ 
