It is known that $$\frac{\partial\log|A|}{\partial A}=A^{-T}$$

However, if $L$ is a lower triangular positive definite matrix and take the log determinant, $\log |L|=\sum_i\log L_{ii}$.

Question is considering that the off diagonal elements of $L$ do not affect the determinant at all why would $$\frac{\partial\log|L|}{\partial L}=L^{-T}$$ $L^{-T}$ has off diagonal terms that are non-zero.

Am I somehow looking at the definition of matrix calculus wrong?


Yes, the problem is with interpretation. If we want to look at the partial derivative of the logdet with respect to a single element $(i,j)$ of $L$, we do this: $$\frac{\partial \log \det L}{\partial L_{ij}} = \left\langle \frac{d \log\det L}{d L}, E_{ij} \right\rangle = \langle L^{-T}, E_{ij} \rangle = \left(L^{-T}\right)_{ij} = \left(L^{-1}\right)_{ji}$$ ($E_{ij}$ is the matrix with a $1$ in the $(i,j)$ position and zeros everywhere else.) When $i>j$, $(L^{-1})_{ji}=0$. So just as expected, perturbations in the strict lower triangle do not change the determinant.

But perturbations in the upper triangle of $L$ certainly do change $\log\det L$! When $i<j$, we expect the partial derivative to be non-zero, typically.

And that is why the matrix derivative involves the entire matrix $L$, not just its diagonal. The vector space over which you are defining this function is the space of all $n\times n$ matrices, including those that are not lower triangular.

If you wanted to, you could define a function just on the vector space of lower triangular matrices: $$f_{LT}:\mathbb{R}^{n\times n}_{LT}\rightarrow\mathbb{R}, \quad f(L) = \log \det L$$ If you do this, then the derivative has to be a lower triangular matrix as well. And since we the know that the strict lower trianglular portion of the derivative is zero, you get a diagonal derivative: $$\frac{d f_{LT}(L)}{d L} = \mathop{\textrm{diag}}(L_{11}^{-1},L_{22}^{-1},\dots,L_{nn}^{-1})$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.