derivative of exponential of matrix trace What is the derivative of $\sum_{ij}e^{-d_{ij}^2(X)}=\sum_{ij}e^{-\operatorname{tr}(X^TC_{ij}X)}$, w.r.t $X$ where $C_{ij}$ is a constant matrix and $d_{ij}^2(X)$ denotes the squared Euclidean distance between the rows $i,j$ of $X$. All the entries here are real
 A: Okay.  It doesn't change much anyhow.  Use linearity of the trace.  Writing $f(X) = {\rm tr}(X^T C_{ij} X)$ and varying $X$ by $\delta X$, we get $f(X+\delta X) - f(X) = {\rm tr}(\delta X^T C_{ij} X) + {\rm tr}(X^T C_{ij} \delta X)$.  Now use what you know about how matrix traces transform under transposition of the argument and also what you know about the form of $C_{ij}$ to simplify that expression and then give the matrix derivative of $g(X)$.
What about the derivative of $g(X) = \exp f(X)$? Since $f$ maps vectors to real numbers, you can use the familiar composition rule on the exponentiation.
You may find that your expression of $C_{ij}$ pulls out components of $X$.  What does the final summation over $i$ and $j$ do?
A: Consider the scalar function
$$ \eqalign{
  f_{ij} &= {\rm exp}(-C_{ij}:XX^T)\cr
}$$
Your objective function is simply the sum of these functions:  $\,\,f=\sum_{ij}f_{ij}$ 
Next, consider the differential of the logarithm of one of these scalar functions
$$ \eqalign{
  {\rm log}(f_{ij}) &= -C_{ij}:XX^T \cr
  d\,{\rm log}(f_{ij}) &= -C_{ij}:d\,(XX^T) \cr
  \frac {df_{ij}}{f_{ij}} &= -C_{ij}:d\,(XX^T) \cr
    &= -2\,C_{ij}\,X:dX \cr
   df_{ij} &= -2\,f_{ij}\,C_{ij}\,X:dX \cr
   \frac {\partial f_{ij}}{\partial X} &= -2\,f_{ij}\,C_{ij}\,X \cr
}$$
The derivative of the objective function is the sum of these derivatives
$$ \eqalign{
   \frac {\partial f}{\partial X} &= -2\,\sum_{ij} \,f_{ij}\,C_{ij}\,X \cr
}$$
You can sum the indexed quantities and collect them into a single matrix $M = \sum_{ij} \,f_{ij}\,C_{ij}$. Now you can write the derivative as $\,\,\frac {\partial f}{\partial X} = -2MX$ 
A: User,  the image of the derivative is a scalar !
Assume that the matrices are real. Moreover, the $(C_{i,j})$ are symmetric matrices. Then the required derivative is 
$H\rightarrow -2\sum_{i,j}Trace(X^TC_{i,j}H)exp(-Trace(X^TC_{i,j}X))$.
