Derivative of a function of matrix I am trying to derive the gradient of the function $f(X) = AXZ + XZX^TXZ$ where $A,X,Z \in R^{n \times n}$ with respect to $X$ matrix. I read a post Matrix-by-matrix derivative formula about matrix derivate, but I am not able to follow it. In my case $\frac{\partial f(X)}{\partial X)}$ would be a tensor, but If I try to use the formula given in the post I would get a matrix. How should I process to get the partial derivate? 
 A: Let
\begin{align}
F := f(X) = AXZ + XZX^TXZ \ .
\end{align}
Now take the differential, then vectorize, and thereafter obtain the gradient.
\begin{align}
dF &= A \ dX \ Z + dX \ Z \ X^T \ X \ Z + X \ Z \ dX^T \ X \ Z + X \ Z \ X^T \ dX \ Z \\ \\
\Longleftrightarrow  \ {\rm vec}\left( dF \right) &= {\rm vec}\left(A \ dX \ Z\right) + {\rm vec}\left(dX \ Z \ \underbrace{X^T \ X \ Z}_{ } \right) \\ &+ {\rm vec}\left(X \ Z \ dX^T \ X \ Z \right)  + {\rm vec}\left(X \ Z \ X^T \ dX \ Z \right) \\ \\
 \Longleftrightarrow  \ {\rm vec}\left( dF \right) &= \left(Z^T \otimes A \right) {\rm vec}\left( dX \right) + \left(\left(X^T X Z\right)^T Z^T \otimes  I\right) {\rm vec}\left( dX\right) \\ 
&+ \left(\left(X Z\right)^T \otimes  \left( XZ\right)\right) \underbrace{{\rm vec} \left( dX^T\right)}_{= K  {\rm vec}\left( dX\right)} + \left(Z^T \otimes  \left( XZX^T\right)\right) {\rm vec}\left( dX\right) \\ \\
\Rightarrow  \frac{\partial f(X)}{dX} =  \frac{\partial {\rm vec}\left( F\right)}{{\rm vec}\left( dX\right)} &= \left(Z^T \otimes A \right)  + \left(\left(X^T X Z\right)^T Z^T \otimes  I\right)  \\ &+ \left(\left(X Z\right)^T \otimes  \left( XZ\right)\right) K  + \left(Z^T \otimes  \left( XZX^T\right)\right) \ , 
\end{align}
where $K$ is the commutation matrix for the Kronecker products.
