# Show that $\frac{\partial}{\partial X}X^{-1} = \left(-X^{-T} \otimes X^{-1}\right)$

Dear Matrix Calculus experts,

Show that $$\frac{\partial}{\partial X}X^{-1} = \left(-X^{-T} \otimes X^{-1}\right)$$ where $\otimes$ is the Kronecker product and $X$ is a square matrix.

My attempt. Do you experts agree? Many thanks in advance.

\begin{align} I &= X^{-1} X \\ \Rightarrow 0 &= dX^{-1} X + X^{-1} dX \\ dX^{-1} &= -X^{-1} dX X^{-1} \end{align}

Now vectorize both sides, i.e., \begin{align} {\rm vec}\left(dX^{-1}\right) &= {\rm vec}\left(-X^{-1} dX X^{-1}\right)\\ &= \left(-X^{-T} \otimes X^{-1}\right) {\rm vec}\left(dX\right) \\ \Rightarrow \underbrace{\frac{\partial}{\partial {\rm vec}(dX)}{\rm vec}\left(dX^{-1}\right)}_{= \frac{\partial}{\partial X}X^{-1} ?} &= \left(-X^{-T} \otimes X^{-1}\right). \end{align}

• The derivation on page 5 of Paul L. Fackler's Notes on Matrix Calculus agrees with what you obtained. So it looks like your desired result is incorrect, unless $X$ is symmetric. – Rahul Jun 26 '18 at 9:18
• I guess there is a typo in the unpublished paper I have received. Probably, my attempt is as expected and matches with the notes you have mentioned. There is no information about the symmetric assumption. – user550103 Jun 26 '18 at 9:23

Allow me state it generally:

If $$dF(X)$$ can be expressed as $$dF(X)=A(dX)B$$, where $$F(X)\in\mathbb{R}^{p\times q}$$ is a matrix function of $$X\in\mathbb{R}^{m\times n}$$, then $$J_XF(X)=B^T\otimes A$$, where the Jacobian matrix $$J_XF(X)$$ is defined as

$$$$J_XF(X)\equiv \frac{\partial vec(F(X))}{\partial (vec X)^T}$$$$

Note the transpose sign on the denominator.

By this rule, given your manipulation $$dX^{-1} = -X^{-1} (dX) X^{-1}$$, we can directly confirm the result that you want to show is correct.

So here we focus on the rule itself.

First vectorize $$dF(X)=A(dX)B$$ on both sides. $$$$d(vec F(X))=vec(A(dX)B)=(B^T\otimes A)vec(dX)=(B^T\otimes A)d(vecX)$$$$ Second we show that $$$$d(vec F(X))=\left[\frac{\partial vec(F(X))}{\partial (vec X)^T}\right] d(vecX) \equiv J_XF(X) d(vecX)$$$$ such that $$J_XF(X)=B^T\otimes A$$, and the rule valids.

To see this, we write $$d(vecF(X))$$ explicitly as follows:

\begin{align} d(vecF(X))=vec(dF(X)) &= \begin{bmatrix} df(X)_{11} \\ \vdots \\ df(X)_{p1} \\ \vdots \\ df(X)_{1q} \\ \vdots \\ df(X)_{pq} \end{bmatrix} \\ &= \begin{bmatrix} \begin{bmatrix} \frac{\partial f(X)_{11}}{\partial x_{11}} \cdots \frac{\partial f(X)_{11}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{11}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{11}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{p1}}{\partial x_{11}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{1q}}{\partial x_{11}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{pq}}{\partial x_{11}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \end{bmatrix} \\ &= \begin{bmatrix} \frac{\partial f(X)_{11}}{\partial x_{11}} \cdots \frac{\partial f(X)_{11}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{11}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{11}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{p1}}{\partial x_{11}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{1q}}{\partial x_{11}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{pq}}{\partial x_{11}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ &= \frac{\partial vec(F(X))}{\partial (vec X)^T} vec(dX) \end{align} where $$df(X)_{kl}$$ is the $$k$$th row, $$l$$th col element of $$dF(X)$$, and it is defined as below, the similar fashion of $$dF(X)$$. $$$$df(X)\equiv\frac{\partial f(X)}{\partial (vecX)^T}d(vecX)$$$$ This completes the proof of the rule.

====================

To clarify the definition of the matrix calculus/differentials: In some textbooks, we use $$J_XF(X)$$ or $$D_XF(X)$$ to express the so-called $$\partial F(X)/\partial X$$. In wikipedia, it corresponds the "numerator layout". While in other materials we use the gradient notation, namely $$\nabla_XF(X)$$, or the "denominator layout". $$\nabla_XF(X)= [J_XF(X)]^T$$. $$$$\nabla_XF(X)\equiv \frac{\partial vec^T F(X)}{\partial vec X}$$$$ Hence, it is entire possible that the results in different textbooks varied by transpose. Note pure $$\partial F(X)/\partial X$$ is not well-defined.