1
$\begingroup$

I am trying to differentiate over the following Frobenius Norm: $$\Phi =||A-(B\circ C)D ||^2_F$$ with respect to B, C, D respectively, i.e.: $$\frac{\partial \Phi}{\partial B}, \frac{\partial \Phi}{\partial C},\frac{\partial \Phi}{\partial D}$$ After expending the norm and writing it in the trace form, it becomes: $$\Phi = tr(A^TA) -tr(A^T(B\circ C)D )-tr(D^T(B^T\circ C^T)A)+tr(D^T(B^T \circ C^T)(B \circ C)D) $$ (please correct me if I did the expansion wrong)

I've tried to write them elements by elements and checked out the whole matrix cookbook.

Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.

Could anyone help me out please? I appreciate a lot.

$\endgroup$
1
$\begingroup$

Define the matrices $$\eqalign{ X &= B\circ C,\quad Y=XD-A \cr }$$ Write the function in terms of these new variables. Then find its differentials and gradients (in alphabetical order). $$\eqalign{ \phi &= \|Y\|_F^2 = Y:Y \cr d\phi &= 2Y:dY \cr &= 2Y:dX\,D \cr &= 2YD^T:C\circ dB \cr &= 2C\circ(YD^T):dB \cr \frac{\partial\phi}{\partial B} &= 2C\circ(YD^T) \cr }$$ Since $B$ and $C$ are interchangeable $$\eqalign{ \frac{\partial\phi}{\partial C} &= 2B\circ(YD^T) \cr }$$ Finally, the gradient wrt $D$ $$\eqalign{ d\phi &= 2Y:X\,dD \cr &= 2X^TY:dD \cr \frac{\partial\phi}{\partial D} &= 2X^TY \cr }$$ NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {\rm Tr}(B^TC)$$

$\endgroup$
1
  • $\begingroup$ Thank you so much for your answer. It is very clear and understandable. This helps me a lot. $\endgroup$
    – Rio Li
    Mar 10 '19 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.