How to calculate the gradient of log det matrix inverse? How to calculate the gradient with respect to $X$ of:
$$
\log \mathrm{det}\, X^{-1}
$$
here $X$ is a positive definite matrix, and det is the determinant of a matrix.
How to calculate this? Or what's the result? Thanks!
 A: I assume that you are asking for the derivative with respect to the elements of the matrix. In this cases first notice that
$$\log \det X^{-1} = \log (\det X)^{-1} = -\log \det X$$
and thus
$$\frac{\partial}{\partial X_{ij}} \log \det X^{-1} = -\frac{\partial}{\partial X_{ij}} \log \det X = - \frac{1}{\det X} \frac{\partial \det X}{\partial X_{ij}} = - \frac{1}{\det X} \mathrm{adj}(X)_{ji} = - (X^{-1})_{ji}$$
since $\mathrm{adj}(X) = \det(X) X^{-1}$ for invertible matrices (where $\mathrm{adj}(X)$ is the adjugate of $X$, see http://en.wikipedia.org/wiki/Adjugate).
A: Or you can check section A.4.1 of the book Stephen Boyd, Lieven Vandenberghe, Convex Optimization for an alternative solution, where they compute the gradient without using the adjugate.
A: The simplest is probably to observe that
$$-\log\det (X+tH) =  -\log\det X -\log\det(I+tX^{-1}H) 
\\= -\log\det X - t \textrm{Tr}(X^{-1}H) + o(t),$$
where is used the "obvious" fact that $\det(I+A) = 1+\textrm{Tr}(A)+o(|A|)$ (all the other terms are quadratic expressions of the coefficients of $A$).
Notice that $\textrm{Tr}(X^{-1}H)=(X^{-T},H)$ in the Frobenius scalar product, hence $\nabla [-\log\det(X)] = -X^{-T}$ in this scalar product. (This gives another proof that $\nabla\det (X) = cof(X)$.)
Of course if $X$ is symmetric positive definite then $-X^{-1}$ is also a valid expression. Moreover, one has in this case, for $X,Y$ positive definite, $(-X^{-1}+Y^{-1},X-Y)\ge 0$.
A: Warning!
The answers given so far work only if $X \in \mathbb{R}^{n\times n}$ is not symmetric and has $n^2$ independent variables! If $X$ is symmetric, then it has only $n(n+1)/2$ independent variables and the correct formula is
$$\frac{\partial \log\det X^{-1}}{\partial X} = -\frac{\partial \log\det X}{\partial X} = -(2X^{-1}-\text{diag}(y_{11}, \dots, y_{nn})),$$
where $y_{ii}$ is the $i$ the entry on the diagonal of $X^{-1}$. This question explains why this is the case.
