1
$\begingroup$

Dear Matrix Calculus experts,

Show that $$\frac{\partial}{\partial X}X^{-1} = \left(-X^{-T} \otimes X^{-1}\right)$$ where $\otimes$ is the Kronecker product and $X$ is a square matrix.


My attempt. Do you experts agree? Many thanks in advance.

\begin{align} I &= X^{-1} X \\ \Rightarrow 0 &= dX^{-1} X + X^{-1} dX \\ dX^{-1} &= -X^{-1} dX X^{-1} \end{align}

Now vectorize both sides, i.e., \begin{align} {\rm vec}\left(dX^{-1}\right) &= {\rm vec}\left(-X^{-1} dX X^{-1}\right)\\ &= \left(-X^{-T} \otimes X^{-1}\right) {\rm vec}\left(dX\right) \\ \Rightarrow \underbrace{\frac{\partial}{\partial {\rm vec}(dX)}{\rm vec}\left(dX^{-1}\right)}_{= \frac{\partial}{\partial X}X^{-1} ?} &= \left(-X^{-T} \otimes X^{-1}\right). \end{align}

$\endgroup$
  • 1
    $\begingroup$ The derivation on page 5 of Paul L. Fackler's Notes on Matrix Calculus agrees with what you obtained. So it looks like your desired result is incorrect, unless $X$ is symmetric. $\endgroup$ – Rahul Jun 26 '18 at 9:18
  • $\begingroup$ I guess there is a typo in the unpublished paper I have received. Probably, my attempt is as expected and matches with the notes you have mentioned. There is no information about the symmetric assumption. $\endgroup$ – user550103 Jun 26 '18 at 9:23
0
$\begingroup$

Allow me state it generally:

If $dF(X)$ can be expressed as $dF(X)=A(dX)B$, where $F(X)\in\mathbb{R}^{p\times q}$ is a matrix function of $X\in\mathbb{R}^{m\times n}$, then $J_XF(X)=B^T\otimes A$, where the Jacobian matrix $J_XF(X)$ is defined as

\begin{equation} J_XF(X)\equiv \frac{\partial vec(F(X))}{\partial (vec X)^T} \end{equation}

Note the transpose sign on the denominator.

By this rule, given your manipulation $dX^{-1} = -X^{-1} (dX) X^{-1}$, we can directly confirm the result that you want to show is correct.

So here we focus on the rule itself.

First vectorize $dF(X)=A(dX)B$ on both sides. \begin{equation} d(vec F(X))=vec(A(dX)B)=(B^T\otimes A)vec(dX)=(B^T\otimes A)d(vecX) \end{equation} Second we show that \begin{equation} d(vec F(X))=\left[\frac{\partial vec(F(X))}{\partial (vec X)^T}\right] d(vecX) \equiv J_XF(X) d(vecX) \end{equation} such that $J_XF(X)=B^T\otimes A$, and the rule valids.

To see this, we write $d(vecF(X))$ explicitly as follows:

\begin{align} d(vecF(X))=vec(dF(X)) &= \begin{bmatrix} df(X)_{11} \\ \vdots \\ df(X)_{p1} \\ \vdots \\ df(X)_{1q} \\ \vdots \\ df(X)_{pq} \end{bmatrix} \\ &= \begin{bmatrix} \begin{bmatrix} \frac{\partial f(X)_{11}}{\partial x_{11}} \cdots \frac{\partial f(X)_{11}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{11}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{11}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{p1}}{\partial x_{11}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{1q}}{\partial x_{11}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ \vdots \\ \begin{bmatrix} \frac{\partial f(X)_{pq}}{\partial x_{11}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \end{bmatrix} \\ &= \begin{bmatrix} \frac{\partial f(X)_{11}}{\partial x_{11}} \cdots \frac{\partial f(X)_{11}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{11}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{11}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{p1}}{\partial x_{11}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{p1}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{1q}}{\partial x_{11}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{1q}}{\partial x_{mn}} \\ \vdots \\ \frac{\partial f(X)_{pq}}{\partial x_{11}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{m1}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{1n}} \cdots \frac{\partial f(X)_{pq}}{\partial x_{mn}} \end{bmatrix} \begin{bmatrix} dx_{11} \\ \vdots \\ dx_{m1} \\ \vdots \\ dx_{1n} \\ \vdots \\ dx_{mn} \end{bmatrix} \\ &= \frac{\partial vec(F(X))}{\partial (vec X)^T} vec(dX) \end{align} where $df(X)_{kl}$ is the $k$th row, $l$th col element of $dF(X)$, and it is defined as below, the similar fashion of $dF(X)$. \begin{equation} df(X)\equiv\frac{\partial f(X)}{\partial (vecX)^T}d(vecX) \end{equation} This completes the proof of the rule.

====================

To clarify the definition of the matrix calculus/differentials: In some textbooks, we use $J_XF(X)$ or $D_XF(X)$ to express the so-called $\partial F(X)/\partial X$. In wikipedia, it corresponds the "numerator layout". While in other materials we use the gradient notation, namely $\nabla_XF(X)$, or the "denominator layout". $\nabla_XF(X)= [J_XF(X)]^T$. \begin{equation} \nabla_XF(X)\equiv \frac{\partial vec^T F(X)}{\partial vec X} \end{equation} Hence, it is entire possible that the results in different textbooks varied by transpose. Note pure $\partial F(X)/\partial X$ is not well-defined.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.