Hessian matrix of the mahalanobis distance wrt the Cholesky decomposition of a covariance matrix

Question

I'm stuck with the following problem: I have to compute the second derivative (hessian matrix) of the mahalanobis distance $$ [x-\mu]^{T} \Sigma^{-1} [x-\mu] $$ wrt to the Cholesky decomposition of the covariance matrix $$ \Lambda\Lambda^{T}=\Sigma $$ On this paper I've found the solution for the first derivative

$$ {\delta J\over\delta\Lambda} = -2 {\Sigma^{-1}} [x-\mu][x-\mu]^{T}\Sigma^{-1}\Lambda $$

Taking the lower triangle of this matrix gives the vector of the first derivatives (gradient) for the k=n*(n+1)/2 components. I haven't been able to find the k*k matrix of second derivatives.

Any help would be appreciated! Thanks!

Answer 1

Let $u=x-\mu$, $f:\Lambda\rightarrow\Lambda\Lambda^T$, $g:\Sigma\rightarrow u^T\Sigma^{-1}u$ and $h=g\circ f$.

1. Then $Dg_{\Sigma}: H\in S_k\rightarrow -u^T\Sigma^{-1}H\Sigma^{-1}u$ and $Df_{\Lambda}:K\in T_k\rightarrow K\Lambda^T+\Lambda K^T$, where $S_k$ is the set of symmetric matrices and $T_k$ is the set of lower triangular matrices. Thus $Dh_{\Lambda}:K\rightarrow -u^T\Sigma^{-1}(K\Lambda^T+\Lambda K^T)\Sigma^{-1}u=-2tr(\Lambda^T\Sigma^{-1}uu^T\Sigma^{-1}K)=-2<\Sigma^{-1}uu^T\Sigma^{-1}\Lambda,K>$

   where $<Y,Z>$ is the real scalar product $tr(Y^TZ)$.

Let $\nabla (h)_{\Lambda}=-2\Sigma^{-1}uu^T\Sigma^{-1}\Lambda$; we'll say that the previous function is the gradient of $h$ because, for every $i\geq j$, $\dfrac{\partial h}{\partial \Lambda_{ij}}=\nabla(h)_{i,j}$; the strictly upper part of $\nabla (h)$ is useless.

2. The second derivative is the following symmetric bilinear form:

$D^2h_{\Lambda}:(K,L)\in T_k\times T_k\rightarrow 2u^T\Sigma^{-1}(L\Lambda^T+\Lambda L^T)\Sigma^{-1}(K\Lambda^T+\Lambda K^T)\Sigma^{-1}u-u^T\Sigma^{-1}(KL^T+LK^T)\Sigma^{-1}u$.

For example, if $p\geq q,r\geq s$, then $\dfrac{\partial^2 h}{\partial \Lambda_{pq}\partial\Lambda_{rs}}=D^2h_{\Lambda}(E_{pq},E_{rs})$.

Note that the Hessian of $h$ is a complicated tensor .