1
$\begingroup$

I have the following function

$$f=\text{Tr(A}^T\text{SA}),$$

where $A$ is a matrix, $S$ is given by

$$S_{ij} = \exp(-\frac{(x_i-x_j)^2}{2\sigma^2})$$


Here is my attempt to calculate the derivative of $f$ w.r.t $X$

We have

$$\begin{align*} f &= \text{Tr}(AA^TS) \\ &= AA^T:S \\ \implies df &=AA^T:dS \\ &= AA^T:(\frac{-1}{2\sigma^2})S\odot dS \\ &= (\frac{-1}{2\sigma^2}) AA^T:S\odot d(diag(X^TX)1^T + 1(diag(X^TX))^T - 2X^TX) \\ &= (\frac{-1}{2\sigma^2}) AA^T:S\odot (diag(2X^TdX)1^T + 1(diag(2X^TdX))^T - 2X^TdX) \\ &= (\frac{-1}{2\sigma^2})(S\odot AA^T): (diag(2X^TdX)1^T + 1(diag(2X^TdX))^T - 2X^TdX) \\ &= (\frac{-1}{2\sigma^2})[-2X(S\odot AA^T):dX + (S\odot AA^T):(diag(2X^TdX)1^T + 1(diag(2X^TdX))^T)] \\ \end{align*}$$

I am stuck with the two terms involving the diag operator,

Could anyone help me to simplify these terms and obtain the final expression of $\frac{df}{dX}$?

Best regards,

$\endgroup$
2
  • 1
    $\begingroup$ Since you're working with diag operators, the following identities may be useful $$\eqalign{ a:{\rm diag}(B) &= {\rm Diag}(a):B \\ (I\odot A):B &= {\rm Diag}({\rm diag}(A)):B \\ &= {\rm diag}(A):{\rm diag}(B) \\ {\rm diag}(a{\tt1}^T) &= {\rm diag}({\tt1}a^T) = a \\ }$$ $\endgroup$
    – greg
    Jul 9 '20 at 0:47
  • $\begingroup$ Very nice identities. Thank you again greg. $\endgroup$ Jul 9 '20 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.