Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What is the derivative of $\sum_{ij}e^{-d_{ij}^2(X)}=\sum_{ij}e^{-\operatorname{tr}(X^TC_{ij}X)}$, w.r.t $X$ where $C_{ij}$ is a constant matrix and $d_{ij}^2(X)$ denotes the squared Euclidean distance between the rows $i,j$ of $X$. All the entries here are real

share|cite|improve this question
You need to clarify your notation a bit. Is $C_{ij}$ a constant matrix for each pair $ij$, i.e. a family of matrices labeled by two indices, $i$ and $j$, or is $C$ a matrix with components $C_{ij}$? If the latter, then there is no additional summation over $i$ and $j$ after evaluating $e^{-{\rm tr}(X^T C X)}$. – josh Jun 5 '13 at 3:45
@josh it is a family of matrices labeled by $i,j$. In fact it is the matrix $C_{ij}$ formed by $(e_i-e_j)(e_i-e_j)^T$ where the $e's$ are the basis vectors. – user75402 Jun 5 '13 at 3:47

Okay. It doesn't change much anyhow. Use linearity of the trace. Writing $f(X) = {\rm tr}(X^T C_{ij} X)$ and varying $X$ by $\delta X$, we get $f(X+\delta X) - f(X) = {\rm tr}(\delta X^T C_{ij} X) + {\rm tr}(X^T C_{ij} \delta X)$. Now use what you know about how matrix traces transform under transposition of the argument and also what you know about the form of $C_{ij}$ to simplify that expression and then give the matrix derivative of $g(X)$.

What about the derivative of $g(X) = \exp f(X)$? Since $f$ maps vectors to real numbers, you can use the familiar composition rule on the exponentiation.

You may find that your expression of $C_{ij}$ pulls out components of $X$. What does the final summation over $i$ and $j$ do?

share|cite|improve this answer
So is it, $-2\sum_{ij}(e^{-tr X^TC_{ij}X})C_{ij}X$ ? If not, please correct it. – user75402 Jun 5 '13 at 4:50
Waiting for a definitive answer.. – user75402 Jun 8 '13 at 0:04

User, the image of the derivative is a scalar ! Assume that the matrices are real. Moreover, the $(C_{i,j})$ are symmetric matrices. Then the required derivative is $H\rightarrow -2\sum_{i,j}Trace(X^TC_{i,j}H)exp(-Trace(X^TC_{i,j}X))$.

share|cite|improve this answer

Consider the scalar function $$ \eqalign{ f_{ij} &= {\rm exp}(-C_{ij}:XX^T)\cr }$$ Your objective function is simply the sum of these functions: $\,\,f=\sum_{ij}f_{ij}$

Next, consider the differential of the logarithm of one of these scalar functions $$ \eqalign{ {\rm log}(f_{ij}) &= -C_{ij}:XX^T \cr d\,{\rm log}(f_{ij}) &= -C_{ij}:d\,(XX^T) \cr \frac {df_{ij}}{f_{ij}} &= -C_{ij}:d\,(XX^T) \cr &= -2\,C_{ij}\,X:dX \cr df_{ij} &= -2\,f_{ij}\,C_{ij}\,X:dX \cr \frac {\partial f_{ij}}{\partial X} &= -2\,f_{ij}\,C_{ij}\,X \cr }$$ The derivative of the objective function is the sum of these derivatives $$ \eqalign{ \frac {\partial f}{\partial X} &= -2\,\sum_{ij} \,f_{ij}\,C_{ij}\,X \cr }$$ You can sum the indexed quantities and collect them into a single matrix $M = \sum_{ij} \,f_{ij}\,C_{ij}$. Now you can write the derivative as $\,\,\frac {\partial f}{\partial X} = -2MX$

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.