# Differentiating mahalanobis distance

I would like to differentiate the mahalanobis distance:

$$D(\textbf{x}, \boldsymbol \mu, \Sigma) = (\textbf{x}-\boldsymbol \mu)^T\Sigma^{-1}(\textbf{x}-\boldsymbol \mu)$$

where $\textbf{x} = (x_1, ..., x_n) \in \mathbb R^n, \;\boldsymbol \mu = (\mu_1, ..., \mu_n) \in \mathbb R^n$ and $$\Sigma = \left( \begin{array}{ccc} E[(X_1-\mu_1)(X_1-\mu_1)] & \cdots & E[(X_1-\mu_1)(X_n-\mu_n)] \\ \vdots & \ddots & \vdots \\ E[(X_n-\mu_n)(X_1-\mu_1)] & \cdots & E[(X_n-\mu_n)(X_n-\mu_n)] \end{array} \right)$$

$\;$

is the covariance matrix. I want to differentiate $D$ with respect to $\boldsymbol\mu$ and $\Sigma$. Can someone show me how to do this? In other words, how to calculate:

$$\frac{\partial D}{\partial \boldsymbol \mu} \;\;\text{and}\;\;\frac{\partial D}{\partial \Sigma}$$? Thnx for any help!

I got the motivation for my question from this source (page 13, EM-algorithm):

http://ptgmedia.pearsoncmg.com/images/0131478249/samplechapter/0131478249_ch03.pdf

For convenience, define the variables \eqalign{ \boldsymbol{z} &= \boldsymbol{x-\mu} \cr \boldsymbol{B} &= \boldsymbol{\Sigma}^{-1} \cr }

and note their differentials \eqalign{ \boldsymbol{dz} &= \boldsymbol{dx = -d\mu} \cr \boldsymbol{dB} &= \boldsymbol{-B \cdot dB^{-1} \cdot B} \cr &= \boldsymbol{-B \cdot d\Sigma \cdot B} \cr }  Next, re-cast your objective function (taking advantage of the symmetry of $\boldsymbol B$) in terms of these variables \eqalign{ D &= \boldsymbol{B:zz} \cr dD &= \boldsymbol{dB:zz + 2B:z\,dz} \cr &= \boldsymbol{zz:dB + 2(B\cdot z)\cdot dz} \cr }

and take derivatives \eqalign{ \frac{\partial D}{\partial \boldsymbol z} &= \boldsymbol{0 + 2(B\cdot z)} \cr \cr \frac{\partial D}{\partial \boldsymbol B} &= \boldsymbol{zz + 0} \cr }  Now use the chain rule to revert to the original variables.

For $\boldsymbol\mu$ we have \eqalign{ dD &= \frac{\partial D}{\partial \boldsymbol z}\cdot \boldsymbol{dz} \cr &= \boldsymbol{2(B\cdot z)\cdot (-d\mu)} \cr \cr \frac{\partial D}{\partial \boldsymbol \mu} &= \boldsymbol{-2(B\cdot z)} \cr &= \boldsymbol{-2\Sigma^{-1}\cdot (x-\mu)} \cr }  And for $\boldsymbol\Sigma$ \eqalign{ dD &= \frac{\partial D}{\partial \boldsymbol B}: \boldsymbol{dB} \cr &= \boldsymbol{zz:(-B\cdot d\Sigma\cdot B)} \cr &= \boldsymbol{(-B\cdot zz\cdot B):(d\Sigma)} \cr \cr \frac{\partial D}{\partial \boldsymbol \Sigma} &= \boldsymbol{-B\cdot zz\cdot B} \cr &= \boldsymbol{-\Sigma^{-1}\cdot (x-\mu)(x-\mu)\cdot \Sigma^{-1}} \cr }

• +1 Thank you for your help =) Could you perhaps elaborate more on why $dz = dx = -d\mu$ and $dB = -B\cdot d\Sigma\cdot B$? Also what does $:$ mean? =) is it division? – jjepsuomi Jan 1 '14 at 21:05
• $\boldsymbol{A:B}$ in index notation is $A_{ik}B_{ik}$; it can also be expressed as $\rm tr(\boldsymbol{A\cdot B^T})$. Given the change-of-variable $\boldsymbol{z \equiv x - y}$, if you hold $\boldsymbol{y}$ constant and allow $\boldsymbol{x}$ to vary, then $\boldsymbol{dz = dx}$. If you take $\boldsymbol{x}$ as constant, then $\boldsymbol{dz = -dy}$ The formula for $\boldsymbol{dB = d\Sigma^{-1}}$ is well known; it can be derived by differentiating the equation $\boldsymbol{\Sigma\cdot \Sigma^{-1} = I}$. – lynne Jan 2 '14 at 0:23
• +1 @lynne Thank you very much =) One last thing to wrap this up, could you give me a link etc. to a source where I can study about this formula $dB = d\Sigma^{-1}$ =) – jjepsuomi Jan 2 '14 at 5:58
• en.wikipedia.org/wiki/… – lynne Jan 2 '14 at 6:27
• @lynne I think that if you expand your answer with full details, it could be a very useful resource (and will be heavily upvoted).. – MadHatter Jun 3 '17 at 14:53