Show that $\frac{\partial}{\partial X}X^{-1} = \left(-X^{-T} \otimes X^{-1}\right)$

Question

Dear Matrix Calculus experts,

Show that $$\frac{\partial}{\partial X}X^{-1} = \left(-X^{-T} \otimes X^{-1}\right)$$ where $\otimes$ is the Kronecker product and $X$ is a square matrix.

---

My attempt. Do you experts agree? Many thanks in advance.

\begin{align} I &= X^{-1} X \\ \Rightarrow 0 &= dX^{-1} X + X^{-1} dX \\ dX^{-1} &= -X^{-1} dX X^{-1} \end{align}

Now vectorize both sides, i.e., \begin{align} {\rm vec}\left(dX^{-1}\right) &= {\rm vec}\left(-X^{-1} dX X^{-1}\right)\\ &= \left(-X^{-T} \otimes X^{-1}\right) {\rm vec}\left(dX\right) \\ \Rightarrow \underbrace{\frac{\partial}{\partial {\rm vec}(dX)}{\rm vec}\left(dX^{-1}\right)}_{= \frac{\partial}{\partial X}X^{-1} ?} &= \left(-X^{-T} \otimes X^{-1}\right). \end{align}

Answer 1

Allow me state it generally:

If $dF(X)$ can be expressed as $dF(X)=A(dX)B$, where $F(X)\in\mathbb{R}^{p\times q}$ is a matrix function of $X\in\mathbb{R}^{m\times n}$, then $J_XF(X)=B^T\otimes A$, where the Jacobian matrix $J_XF(X)$ is defined as

\begin{equation} J_XF(X)\equiv \frac{\partial vec(F(X))}{\partial (vec X)^T} \end{equation}

Note the transpose sign on the denominator.

By this rule, given your manipulation $dX^{-1} = -X^{-1} (dX) X^{-1}$, we can directly confirm the result that you want to show is correct.

So here we focus on the rule itself.

First vectorize $dF(X)=A(dX)B$ on both sides. \begin{equation} d(vec F(X))=vec(A(dX)B)=(B^T\otimes A)vec(dX)=(B^T\otimes A)d(vecX) \end{equation} Second we show that \begin{equation} d(vec F(X))=\left[\frac{\partial vec(F(X))}{\partial (vec X)^T}\right] d(vecX) \equiv J_XF(X) d(vecX) \end{equation} such that $J_XF(X)=B^T\otimes A$, and the rule valids.

To see this, we write $d(vecF(X))$ explicitly as follows:

\begin{align} d(vecF(X))=vec(dF(X)) &= \begin{bmatrix} df(X)_{11} \\ \vdots \\ df(X)_{p1} \\ \vdots \\ df(X)_{1q} \\ \vdots \\ df(X)_{pq} \end{bmatrix} \\ &= \begin{bmatrix} \begin{bmatrix} \frac{\partial f(X)_{11}}{\partial x_{11}} \cdots \frac{\partial f(X)_{11}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{11}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{11}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{p1}}{\partial x_{11}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{1q}}{\partial x_{11}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{pq}}{\partial x_{11}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \end{bmatrix} \\ &= \begin{bmatrix} \frac{\partial f(X)_{11}}{\partial x_{11}} \cdots \frac{\partial f(X)_{11}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{11}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{11}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{p1}}{\partial x_{11}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{1q}}{\partial x_{11}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{pq}}{\partial x_{11}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ &= \frac{\partial vec(F(X))}{\partial (vec X)^T} vec(dX) \end{align} where $df(X)_{kl}$ is the $k$th row, $l$th col element of $dF(X)$, and it is defined as below, the similar fashion of $dF(X)$. \begin{equation} df(X)\equiv\frac{\partial f(X)}{\partial (vecX)^T}d(vecX) \end{equation} This completes the proof of the rule.

====================

To clarify the definition of the matrix calculus/differentials: In some textbooks, we use $J_XF(X)$ or $D_XF(X)$ to express the so-called $\partial F(X)/\partial X$. In wikipedia, it corresponds the "numerator layout". While in other materials we use the gradient notation, namely $\nabla_XF(X)$, or the "denominator layout". $\nabla_XF(X)= [J_XF(X)]^T$. \begin{equation} \nabla_XF(X)\equiv \frac{\partial vec^T F(X)}{\partial vec X} \end{equation} Hence, it is entire possible that the results in different textbooks varied by transpose. Note pure $\partial F(X)/\partial X$ is not well-defined.