# second order derivative of log det of matrix

I need to find the second order derivative of a matrix.

$f(\pmb{\Delta})=\log \det(\pmb{I}+k\pmb{V^T \Delta V})$

Where $\pmb{\Delta}$ is a triangular matrix.

I did the first order derivative but not sure if is correct. $$\frac{\partial f(\pmb{\Delta})}{\partial [\pmb{\Delta}]_{ii}} = \left[ k\pmb{V} \left( \pmb{I}+k\pmb{V^H}\pmb{\Delta}\pmb{V} \right)^{-1}\pmb{V^H} \right]_{ii}$$

This is correct? How about the second order derivative?

• I get the same derivative at least. Now you want to derive this again with respect to $\Delta_{jj}$? – amsmath Sep 3 '18 at 22:08
• I believe i need to derive again with respect to $\Delta_{ii}$ – Carlos Sep 3 '18 at 22:16
• Setting $k=1$ I get $-[V(I+V^T\Delta V)^{-1}a_ia_i^T(I+V^T\Delta V)^{-1}V^T]_{ii}$, with $a_i = V^Te_i$, where $e_i$ is the $i$-th standard basis vector. – amsmath Sep 4 '18 at 1:48

Define the matrix variables \eqalign{ X &= \Delta \cr A &= I + kV^TXV \implies dA &= kV^T\,dX\,V \cr Y &= kVA^{-T}V^T \cr } Write the function in terms of these variables and calculate its differential and gradient. \eqalign{ f &= \log\det A \cr df &= A^{-T}:dA \cr &= A^{-T}:kV^T\,dX\,V \cr &= kVA^{-T}V^T:dX \cr G = \frac{\partial f}{\partial X} &= kVA^{-T}V^T \cr } Repeat the process for the gradient. \eqalign{ dG &= kV\,dA^{-T}\,V^T \cr &= -kVA^{-T}\,dA^T\,A^{-T}V^T \cr &= -k^2VA^{-T}V^T\,dX^T\,VA^{-T}V^T \cr &= -Y\,dX^T\,Y \cr &= -Y{\mathcal E}Y:{\mathcal L}:dX \cr {\mathcal H} = \frac{\partial G}{\partial X} &= -Y{\mathcal E}Y:{\mathcal L} \cr } where $({\mathcal E},{\mathcal L})$ are 4th order tensors whose components can be written in terms of Kronecker deltas \eqalign{ {\mathcal E}_{ijkl} &= {\delta}_{ik} \, {\delta}_{jl} \cr {\mathcal L}_{ijkl} &= {\delta}_{il} \, {\delta}_{jk} \cr } Note that the second derivative (aka Hessian) is itself a 4th order tensor.

Double and single contraction products were used in some of the steps above, which can be expressed in component form as \eqalign{ {\mathcal B}_{ikln} &= \big(-Y{\mathcal E}Y\big)_{ikln} &= \sum_j\sum_m -Y_{ij}{\mathcal E}_{jklm}Y_{mn} \cr {\mathcal H}_{ijmn} &= \big({\mathcal B}:{\mathcal L}\big)_{ijmn} &= \sum_k\sum_l {\mathcal B}_{ijkl}{\mathcal L}_{klmn} \cr A:X &= \sum_i\sum_j A_{ij}X_{ij} &= {\,\rm Tr}\big(A^TX\big) \cr }

• do you have a good reference to "tensor calculus" and how to use it with matrix derivatives/calculus? Thank you in advance. – user550103 Sep 4 '18 at 13:27
• For making $f = \log\det A$ and $df = A^{-T}:dA$ you need to assume that $A$ is invertible right? – Carlos Sep 4 '18 at 21:46
• @Carlos Your question is predicated upon the fact that $A$ is invertible. If it's not, then $\det A=0$ and the logarithm is undefined. – greg Sep 4 '18 at 21:53
• Any way to solve this without tensor? – Carlos Sep 4 '18 at 21:54
• @Carlos As I tried to show in the last few lines of my post, you can write the Hessian in index notation, i.e. $${\mathcal H}_{ijmn}=\frac{\partial G_{ij}}{\partial X_{mn}}= -Y_{in}Y_{ml}$$ – greg Sep 5 '18 at 2:56