Gradient of Frobenius norm of a diagonal matrix

Question

I'd like to find the Gradient of the Frobenius Norm of this diagonal matrix, in respect to $\Phi$:

$\lVert diag(\Phi^tW\Phi) \rVert_F^2$

Is an analytical solution even exists ?

Thanks.

Answer 1

First, we need notation for some non-standard matrix products.

Let $$\eqalign{ A\circ B \cr A:B&={\rm tr}(A^TB) }$$ denote the elementwise/Hadamard and inner/Frobenius product, respectively.

Next, define the matrices $$\eqalign{ A &= \Phi^TW\Phi \cr X &= {\rm Diag}(A) = I\circ A \cr\cr }$$

Write the function in terms of these products and matrices. Then find its differential and gradient. $$\eqalign{ f &= X:X \cr \cr df &= 2X:dX \cr &= 2(I\circ A):(I\circ dA) \cr &= 2(I\circ A):dA \cr &= 2(I\circ A):(d\Phi^T\,W\Phi+\Phi^TW\,d\Phi) \cr &= 2(W+W^T)\Phi(I\circ A):d\Phi \cr \cr \frac{\partial f}{\partial\Phi} &= 2(W+W^T)\Phi(I\circ A) \cr &= 2\,(W+W^T)\,\Phi\,{\rm Diag}(\Phi^TW\Phi) \cr \cr }$$