# Matrix Differentiation (involving Hadamard products)

I am trying to differentiate over the following Frobenius Norm: $$\Phi =||A-(B\circ C)D ||^2_F$$ with respect to B, C, D respectively, i.e.: $$\frac{\partial \Phi}{\partial B}, \frac{\partial \Phi}{\partial C},\frac{\partial \Phi}{\partial D}$$ After expending the norm and writing it in the trace form, it becomes: $$\Phi = tr(A^TA) -tr(A^T(B\circ C)D )-tr(D^T(B^T\circ C^T)A)+tr(D^T(B^T \circ C^T)(B \circ C)D)$$ (please correct me if I did the expansion wrong)

I've tried to write them elements by elements and checked out the whole matrix cookbook.

Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.

Could anyone help me out please? I appreciate a lot.

Define the matrices \eqalign{ X &= B\circ C,\quad Y=XD-A \cr } Write the function in terms of these new variables. Then find its differentials and gradients (in alphabetical order). \eqalign{ \phi &= \|Y\|_F^2 = Y:Y \cr d\phi &= 2Y:dY \cr &= 2Y:dX\,D \cr &= 2YD^T:C\circ dB \cr &= 2C\circ(YD^T):dB \cr \frac{\partial\phi}{\partial B} &= 2C\circ(YD^T) \cr } Since $$B$$ and $$C$$ are interchangeable \eqalign{ \frac{\partial\phi}{\partial C} &= 2B\circ(YD^T) \cr } Finally, the gradient wrt $$D$$ \eqalign{ d\phi &= 2Y:X\,dD \cr &= 2X^TY:dD \cr \frac{\partial\phi}{\partial D} &= 2X^TY \cr } NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {\rm Tr}(B^TC)$$