# Derivative of trace of inverse matrix?

I've been trying to derive the formula for the derivative of $Tr(X^{-1})$ w.r.t. $X$, which I know is $X^{-2T}$. According to the Matrix Cookbook $$\dfrac{\partial g(U)}{\partial X_{ij}} = \operatorname{Tr}\left[\left(\dfrac{\partial g(U)}{\partial U}\right)^T \dfrac{\partial U}{\partial X_{ij}}\right]$$ So, if we let $U = X^{-1}$ and $g(X) = \operatorname{Tr}[X]$, we get $$\dfrac{\partial \,\operatorname{Tr}(X^{-1})}{\partial X_{ij}} = \operatorname{Tr}\left[\left(\dfrac{\partial \,\operatorname{Tr}(U)}{\partial U}\right)^T \dfrac{\partial X^{-1}}{\partial X_{ij}}\right] = \operatorname{Tr}\left[ \dfrac{\partial X^{-1}}{\partial X_{ij}}\right]$$ since the derivative of the trace of a matrix w.r.t. that matrix is just the identity. However, I'm not sure how to proceed from here. I know that the derivative of the inverse of a matrix w.r.t. that matrix is $-X^{-2}$, but I don't know what the derivative of the inverse w.r.t. a specific entry would be, nor do I know what the trace of that derivative matrix would be. Is there something I'm missing here?

This derivatives for matrix functions are better handled as directional derivatives. Denote $h(X)=\text{tr}(X^{-1})$. We have: $$dh(X)(U)=\lim_{t\to 0}\frac{h(X+tU)-h(X)}{t}=\lim_{t\to 0}\frac{\text{tr}((X+tU)^{-1})-\text{tr}(X^{-1})}{t}$$ $$=\text{tr}\Big(\lim_{t\to 0}\frac{(X+tU)^{-1}-(X^{-1})}{t}\Big) =\text{tr}\Big(\lim_{t\to 0}(X+tU)^{-1}\frac{X-(X+tU)}{t}X^{-1}\Big)$$ $$=\text{tr}\Big(\lim_{t\to 0}(X+tU)^{-1}(-U)X^{-1}\Big)=-\text{tr}(X^{-1}UX^{-1}).$$
If one wants to use the formulae in the Matrix Cookbook (that follow all using directional derivatives as above), then use the chain rule in the form $\partial g(X)=\partial g(\partial X)$. In our case: $$\partial(\text{tr}(X^{-1})=\text{tr}(\partial(X^{-1}))=\text{tr}(-X^{-1}(\partial X)X^{-1})= -\text{tr}(X^{-1}(\partial X)X^{-1})$$ (derivative of the trace is the trace itself).
• I see, but why is your final answer $−tr(X^{−1}UX^{−1})$? It should be $X^{-2T}$. Even if you divide by $U$ (which I don't know why you can do even though there is a $U$ on both sides) you get a scalar, when the final answer should be a matrix. – user2258552 Feb 10 '15 at 13:36
• The point is not to divide by $U$, which indeed can't be done. One takes $U=U_{ij}$, the matrix with all entries zero except $1$ at the $(i,j)$ place. This substitution gives the $(i,j)$ entry of $\partial(\text{tr}(X^{-1}))$. Making the (boring) computations one checks finally that $$-\text{tr}(X^{-1}U_{ij}X^{-1})=-[X^{-2}]_{ji}.$$ – Jesus RS Feb 10 '15 at 15:32
• No need to do the boring stuff, use the cyclic property of trace to write $tr(X^{-1}UX^{-1}) = tr(X^{-1}X^{-1}U) = X^{-2T} \cdot U$. The LHS can also be written as this matrix inner product if you interpret dh as the gradient $\nabla h$ so that you have $\nabla h \cdot U = X^{-2T} \cdot U$ for all $U$. Hence proved. – me10240 Nov 20 '17 at 16:52