# Hessian matrix of the mahalanobis distance wrt the Cholesky decomposition of a covariance matrix

I'm stuck with the following problem: I have to compute the second derivative (hessian matrix) of the mahalanobis distance $$[x-\mu]^{T} \Sigma^{-1} [x-\mu]$$ wrt to the Cholesky decomposition of the covariance matrix $$\Lambda\Lambda^{T}=\Sigma$$ On this paper I've found the solution for the first derivative

$${\delta J\over\delta\Lambda} = -2 {\Sigma^{-1}} [x-\mu][x-\mu]^{T}\Sigma^{-1}\Lambda$$

Taking the lower triangle of this matrix gives the vector of the first derivatives (gradient) for the k=n*(n+1)/2 components. I haven't been able to find the k*k matrix of second derivatives.

Any help would be appreciated! Thanks!

Let $$u=x-\mu$$, $$f:\Lambda\rightarrow\Lambda\Lambda^T$$, $$g:\Sigma\rightarrow u^T\Sigma^{-1}u$$ and $$h=g\circ f$$.

1. Then $$Dg_{\Sigma}: H\in S_k\rightarrow -u^T\Sigma^{-1}H\Sigma^{-1}u$$ and $$Df_{\Lambda}:K\in T_k\rightarrow K\Lambda^T+\Lambda K^T$$, where $$S_k$$ is the set of symmetric matrices and $$T_k$$ is the set of lower triangular matrices. Thus $$Dh_{\Lambda}:K\rightarrow -u^T\Sigma^{-1}(K\Lambda^T+\Lambda K^T)\Sigma^{-1}u=-2tr(\Lambda^T\Sigma^{-1}uu^T\Sigma^{-1}K)=-2<\Sigma^{-1}uu^T\Sigma^{-1}\Lambda,K>$$

where $$$$ is the real scalar product $$tr(Y^TZ)$$.

Let $$\nabla (h)_{\Lambda}=-2\Sigma^{-1}uu^T\Sigma^{-1}\Lambda$$; we'll say that the previous function is the gradient of $$h$$ because, for every $$i\geq j$$, $$\dfrac{\partial h}{\partial \Lambda_{ij}}=\nabla(h)_{i,j}$$; the strictly upper part of $$\nabla (h)$$ is useless.

1. The second derivative is the following symmetric bilinear form:

$$D^2h_{\Lambda}:(K,L)\in T_k\times T_k\rightarrow 2u^T\Sigma^{-1}(L\Lambda^T+\Lambda L^T)\Sigma^{-1}(K\Lambda^T+\Lambda K^T)\Sigma^{-1}u-u^T\Sigma^{-1}(KL^T+LK^T)\Sigma^{-1}u$$.

For example, if $$p\geq q,r\geq s$$, then $$\dfrac{\partial^2 h}{\partial \Lambda_{pq}\partial\Lambda_{rs}}=D^2h_{\Lambda}(E_{pq},E_{rs})$$.

Note that the Hessian of $$h$$ is a complicated tensor .

• Thanks for the answer, it is of great help. I came with a similar solution (i.e. using the chain rule), even if I did not formulate it so nicely. For more information on this topic, people that are interested can check this paper from Thomas Minka research.microsoft.com/en-us/um/people/minka/papers/matrix/… – Vincent Moens Jul 11 '16 at 10:31