1
$\begingroup$

I need to find the second order derivative of a matrix.

$f(\pmb{\Delta})=\log \det(\pmb{I}+k\pmb{V^T \Delta V})$

Where $ \pmb{\Delta}$ is a triangular matrix.

I did the first order derivative but not sure if is correct. $$\frac{\partial f(\pmb{\Delta})}{\partial [\pmb{\Delta}]_{ii}} = \left[ k\pmb{V} \left( \pmb{I}+k\pmb{V^H}\pmb{\Delta}\pmb{V} \right)^{-1}\pmb{V^H} \right]_{ii} $$

This is correct? How about the second order derivative?

$\endgroup$
  • $\begingroup$ I get the same derivative at least. Now you want to derive this again with respect to $\Delta_{jj}$? $\endgroup$ – amsmath Sep 3 '18 at 22:08
  • $\begingroup$ I believe i need to derive again with respect to $\Delta_{ii}$ $\endgroup$ – Carlos Sep 3 '18 at 22:16
  • $\begingroup$ Setting $k=1$ I get $-[V(I+V^T\Delta V)^{-1}a_ia_i^T(I+V^T\Delta V)^{-1}V^T]_{ii}$, with $a_i = V^Te_i$, where $e_i$ is the $i$-th standard basis vector. $\endgroup$ – amsmath Sep 4 '18 at 1:48
2
$\begingroup$

Define the matrix variables $$\eqalign{ X &= \Delta \cr A &= I + kV^TXV \implies dA &= kV^T\,dX\,V \cr Y &= kVA^{-T}V^T \cr }$$ Write the function in terms of these variables and calculate its differential and gradient. $$\eqalign{ f &= \log\det A \cr df &= A^{-T}:dA \cr &= A^{-T}:kV^T\,dX\,V \cr &= kVA^{-T}V^T:dX \cr G = \frac{\partial f}{\partial X} &= kVA^{-T}V^T \cr }$$ Repeat the process for the gradient. $$\eqalign{ dG &= kV\,dA^{-T}\,V^T \cr &= -kVA^{-T}\,dA^T\,A^{-T}V^T \cr &= -k^2VA^{-T}V^T\,dX^T\,VA^{-T}V^T \cr &= -Y\,dX^T\,Y \cr &= -Y{\mathcal E}Y:{\mathcal L}:dX \cr {\mathcal H} = \frac{\partial G}{\partial X} &= -Y{\mathcal E}Y:{\mathcal L} \cr }$$ where $({\mathcal E},{\mathcal L})$ are 4th order tensors whose components can be written in terms of Kronecker deltas $$\eqalign{ {\mathcal E}_{ijkl} &= {\delta}_{ik} \, {\delta}_{jl} \cr {\mathcal L}_{ijkl} &= {\delta}_{il} \, {\delta}_{jk} \cr }$$ Note that the second derivative (aka Hessian) is itself a 4th order tensor.

Double and single contraction products were used in some of the steps above, which can be expressed in component form as $$\eqalign{ {\mathcal B}_{ikln} &= \big(-Y{\mathcal E}Y\big)_{ikln} &= \sum_j\sum_m -Y_{ij}{\mathcal E}_{jklm}Y_{mn} \cr {\mathcal H}_{ijmn} &= \big({\mathcal B}:{\mathcal L}\big)_{ijmn} &= \sum_k\sum_l {\mathcal B}_{ijkl}{\mathcal L}_{klmn} \cr A:X &= \sum_i\sum_j A_{ij}X_{ij} &= {\,\rm Tr}\big(A^TX\big) \cr }$$

$\endgroup$
  • $\begingroup$ do you have a good reference to "tensor calculus" and how to use it with matrix derivatives/calculus? Thank you in advance. $\endgroup$ – user550103 Sep 4 '18 at 13:27
  • $\begingroup$ For making $f = \log\det A$ and $ df = A^{-T}:dA $ you need to assume that $A$ is invertible right? $\endgroup$ – Carlos Sep 4 '18 at 21:46
  • 1
    $\begingroup$ @Carlos Your question is predicated upon the fact that $A$ is invertible. If it's not, then $\det A=0$ and the logarithm is undefined. $\endgroup$ – greg Sep 4 '18 at 21:53
  • $\begingroup$ Any way to solve this without tensor? $\endgroup$ – Carlos Sep 4 '18 at 21:54
  • $\begingroup$ @Carlos As I tried to show in the last few lines of my post, you can write the Hessian in index notation, i.e. $${\mathcal H}_{ijmn}=\frac{\partial G_{ij}}{\partial X_{mn}}= -Y_{in}Y_{ml}$$ $\endgroup$ – greg Sep 5 '18 at 2:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.