3
$\begingroup$

I've been trying to derive the formula for the derivative of $Tr(X^{-1})$ w.r.t. $X$, which I know is $X^{-2T}$. According to the Matrix Cookbook $$\dfrac{\partial g(U)}{\partial X_{ij}} = \operatorname{Tr}\left[\left(\dfrac{\partial g(U)}{\partial U}\right)^T \dfrac{\partial U}{\partial X_{ij}}\right]$$ So, if we let $U = X^{-1}$ and $g(X) = \operatorname{Tr}[X]$, we get $$\dfrac{\partial \,\operatorname{Tr}(X^{-1})}{\partial X_{ij}} = \operatorname{Tr}\left[\left(\dfrac{\partial \,\operatorname{Tr}(U)}{\partial U}\right)^T \dfrac{\partial X^{-1}}{\partial X_{ij}}\right] = \operatorname{Tr}\left[ \dfrac{\partial X^{-1}}{\partial X_{ij}}\right]$$ since the derivative of the trace of a matrix w.r.t. that matrix is just the identity. However, I'm not sure how to proceed from here. I know that the derivative of the inverse of a matrix w.r.t. that matrix is $-X^{-2}$, but I don't know what the derivative of the inverse w.r.t. a specific entry would be, nor do I know what the trace of that derivative matrix would be. Is there something I'm missing here?

$\endgroup$
5
$\begingroup$

This derivatives for matrix functions are better handled as directional derivatives. Denote $h(X)=\text{tr}(X^{-1})$. We have: $$ dh(X)(U)=\lim_{t\to 0}\frac{h(X+tU)-h(X)}{t}=\lim_{t\to 0}\frac{\text{tr}((X+tU)^{-1})-\text{tr}(X^{-1})}{t} $$ $$ =\text{tr}\Big(\lim_{t\to 0}\frac{(X+tU)^{-1}-(X^{-1})}{t}\Big) =\text{tr}\Big(\lim_{t\to 0}(X+tU)^{-1}\frac{X-(X+tU)}{t}X^{-1}\Big) $$ $$ =\text{tr}\Big(\lim_{t\to 0}(X+tU)^{-1}(-U)X^{-1}\Big)=-\text{tr}(X^{-1}UX^{-1}). $$

If one wants to use the formulae in the Matrix Cookbook (that follow all using directional derivatives as above), then use the chain rule in the form $\partial g(X)=\partial g(\partial X)$. In our case: $$ \partial(\text{tr}(X^{-1})=\text{tr}(\partial(X^{-1}))=\text{tr}(-X^{-1}(\partial X)X^{-1})= -\text{tr}(X^{-1}(\partial X)X^{-1}) $$ (derivative of the trace is the trace itself).

$\endgroup$
  • $\begingroup$ In any case, thanks for the Matrix Cookbook! $\endgroup$ – Jesus RS Feb 10 '15 at 10:31
  • $\begingroup$ I see, but why is your final answer $−tr(X^{−1}UX^{−1})$? It should be $X^{-2T}$. Even if you divide by $U$ (which I don't know why you can do even though there is a $U$ on both sides) you get a scalar, when the final answer should be a matrix. $\endgroup$ – user2258552 Feb 10 '15 at 13:36
  • 2
    $\begingroup$ The point is not to divide by $U$, which indeed can't be done. One takes $U=U_{ij}$, the matrix with all entries zero except $1$ at the $(i,j)$ place. This substitution gives the $(i,j)$ entry of $\partial(\text{tr}(X^{-1}))$. Making the (boring) computations one checks finally that $$-\text{tr}(X^{-1}U_{ij}X^{-1})=-[X^{-2}]_{ji}.$$ $\endgroup$ – Jesus RS Feb 10 '15 at 15:32
  • $\begingroup$ Still, it's a nice compressed way of expressing these formulas! $\endgroup$ – Jesus RS Feb 10 '15 at 15:35
  • $\begingroup$ No need to do the boring stuff, use the cyclic property of trace to write $tr(X^{-1}UX^{-1}) = tr(X^{-1}X^{-1}U) = X^{-2T} \cdot U$. The LHS can also be written as this matrix inner product if you interpret dh as the gradient $\nabla h$ so that you have $\nabla h \cdot U = X^{-2T} \cdot U$ for all $U$. Hence proved. $\endgroup$ – me10240 Nov 20 '17 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.