I have a matrix kernel function which I am trying to find the derivative to. Function is K = c * exp[-1/2 * (P(X1 - X2))' * P(X1 -X2)] where uppercase are matrices and lower case are scalars (and ' denotes transpose). I'm trying to find dK/dP. I'm pretty rusty on matrix calculus, can anyone give me a hand here?


  • 1
    $\begingroup$ Your notation is not very clear. Is $P$ a matrix? then did you mean something like $K(P) = c \exp[-\frac12 \| P(X_1 - X_2) \|^2]$? In that case you should probably write (P(X1-X2))', the transpose should be around everything and maybe a trace before it? Is P symmetric or not? $\endgroup$ – passerby51 Jun 13 '12 at 17:43
  • $\begingroup$ I was assuming this is a matrix valued function. If that's true I'd undelete my (by now corrected) answer. If @passerby51 assumption is correct, the derivative is just $$ D_V K = -1/2K \langle V(X_1-X_2), P(X_1-X_2)\rangle$$ with $\langle.,.\rangle$ the scalar product on the vector space of matrices. $\endgroup$ – user20266 Jun 13 '12 at 17:50
  • $\begingroup$ Sorry yeah the transpose is around the whole thing. P is not symmetric. $\endgroup$ – tomas Jun 13 '12 at 17:50
  • $\begingroup$ Yeah it is a matrix valued function. I didn't quite get a chance to see your deleted answer Thomas. What was it? Thanks for the response guys. $\endgroup$ – tomas Jun 13 '12 at 17:53
  • $\begingroup$ undeleted my reply. $\endgroup$ – user20266 Jun 13 '12 at 17:53

Let $$Z(P)=(P(X_1-X_2))^TP(X_1-X_2)$$ If $K$ is viewed as depending only on $P$ and if you differentiate in direction $V$ you get $$ D_V K = K*\frac{-1}{2} \left\{\frac{I-e^{-ad_{Z(P)}}}{ad_{Z(p)}} \left[ (V(X_1-X_2))^TP(X_1-X_2) + P(X_1-X_2)^TV(X_1-X_2) \right] \right\}$$

The term on the right hand side in curly parenthesis needs explanation. It is the matrix valued function $\frac{I-e^{-ad_{Z(P)}}}{ad_{Z(P))}}$ applied to the term in square brackets. This in turn means

$$\frac{I-e^{-ad_{Z}}}{ad_{Z}}[Y] = Y-\frac{[Z,Y]}{2!} + \frac{[Z,[Z,Y]]}{3!} - \ldots$$

See, e.g., Chapter 3.3 in Brian C. Halls Book 'Lie Groups, Lie Algebras and Representations for a derivation of the derivative of the exponential.

(Sorry for posting a too simple and wrong answer first, which is true only if $Z$ and $D_VZ $ commute). (I don't like the $\frac{d}{dP}$ notation).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.