I'd like to find the Gradient of the Frobenius Norm of this diagonal matrix, in respect to $\Phi$:
$\lVert diag(\Phi^tW\Phi) \rVert_F^2$
Is an analytical solution even exists ?
Thanks.
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this communityFirst, we need notation for some non-standard matrix products.
Let $$\eqalign{ A\circ B \cr A:B&={\rm tr}(A^TB) }$$ denote the elementwise/Hadamard and inner/Frobenius product, respectively.
Next, define the matrices
$$\eqalign{
A &= \Phi^TW\Phi \cr
X &= {\rm Diag}(A) = I\circ A \cr\cr
}$$
Write the function in terms of these products and matrices. Then find its differential and gradient. $$\eqalign{ f &= X:X \cr \cr df &= 2X:dX \cr &= 2(I\circ A):(I\circ dA) \cr &= 2(I\circ A):dA \cr &= 2(I\circ A):(d\Phi^T\,W\Phi+\Phi^TW\,d\Phi) \cr &= 2(W+W^T)\Phi(I\circ A):d\Phi \cr \cr \frac{\partial f}{\partial\Phi} &= 2(W+W^T)\Phi(I\circ A) \cr &= 2\,(W+W^T)\,\Phi\,{\rm Diag}(\Phi^TW\Phi) \cr \cr }$$