# Derivative of the inverse of a matrix

In a scientific paper, I've seen the following

$$\frac{\delta K^{-1}}{\delta p} = -K^{-1}\frac{\delta K}{\delta p}K^{-1}$$

where $$K$$ is a $$n \times n$$ matrix that depends on $$p$$. In my calculations I would have done the following

$$\frac{\delta K^{-1}}{\delta p} = -K^{-2}\frac{\delta K}{\delta p}=-K^{-T}K^{-1}\frac{\delta K}{\delta p}$$

Is my calculation wrong?

Note: I think $$K$$ is symmetric.

• It is wrong. Note: matrices do not commute in general. Respect the order when derivate.
– A.Γ.
Commented Oct 9, 2015 at 12:05
• Hint:$\partial_{\rho}(KK^{-1})=\partial_{\rho}\mathbb{1}$ Commented Oct 9, 2015 at 12:05
• @tired what is $\mathbb{1}$? Do you perhaps mean $I$? Commented Sep 9, 2021 at 8:56

The major trouble in matrix calculus is that the things are no longer commuting, but one tends to use formulae from the scalar function calculus like $(x(t)^{-1})'=-x(t)^{-2}x'(t)$ replacing $x$ with the matrix $K$. One has to be more careful here and pay attention to the order. The easiest way to get the derivative of the inverse is to derivate the identity $I=KK^{-1}$ respecting the order $$\underbrace{(I)'}_{=0}=(KK^{-1})'=K'K^{-1}+K(K^{-1})'.$$ Solving this equation with respect to $(K^{-1})'$ (again paying attention to the order (!)) will give $$K(K^{-1})'=-K'K^{-1}\qquad\Rightarrow\qquad (K^{-1})'=-K^{-1}K'K^{-1}.$$

• Looks like this also holds for total derivative with p being any parameter set, following the same derivation Commented Jul 15, 2019 at 11:16

Yes, your calculation is wrong, note that $K$ may not commute with $\frac{\partial K}{\partial p}$, hence you must apply the chain rule correctly. The derivative of $\def\inv{\mathrm{inv}}\inv \colon \def\G{\mathord{\rm GL}}\G_n \to \G_n$ is not given by $\inv'(A)B = -A^2B$, but by $\inv'(A)B = -A^{-1}BA^{-1}$. To see that, note that for small enough $B$ we have \begin{align*} \inv(A + B) &= (A + B)^{-1}\\ &= (\def\I{\mathord{\rm Id}}\I + A^{-1}B)^{-1}A^{-1}\\ &= \sum_k (-1)^k (A^{-1}B)^kA^{-1}\\ &= A^{-1} - A^{-1}BA^{-1} + o(\|B\|) \end{align*} Hence, $\inv'(A)B= -A^{-1}BA^{-1}$, and therefore, by the chain rule $$\partial_p (\inv \circ K) = \inv'\circ K\bigl(\partial_p K) = -K^{-1}(\partial_p K) K^{-1}$$

• How does the second line follow from B being small enough? Commented Apr 30, 2018 at 13:19
• @nbubis The Neumann series is $I - A = \sum_{k = 0}^{\infty} A^k$ for $\| A \| < 1$, which is analogous to the geometric series. Thus for small enough $B$ we have $\| A^{-1} B \| < 1$ and thus $(I + A^{-1} B)^{-1} = \sum_{k = 0}^{\infty} (-1)^k (A^{-k} B)^k$. Commented May 28, 2020 at 22:34

Actually, we can directly calculate the derivate of a matrix starting from the definition of the derivate of functions. In particular, \begin{align} \frac{dK^{-1}}{dp} & =\lim_{\Delta p \to 0} \frac{(K+\Delta K)^{-1} - K^{-1}}{\Delta p} \\ {} & = \lim_{\Delta p \to 0} \frac{(K+\Delta K)^{-1}KK^{-1} - (K+\Delta K)^{-1}(K+\Delta K)K^{-1}}{\Delta p} \\ {} & = \lim_{\Delta p \to 0} \frac{(K+\Delta K)^{-1}(-\Delta K) K^{-1}}{\Delta p} \\ {} & = - K^{-1} \lim_{\Delta p \to 0} \frac{\Delta K}{\Delta p} K^{-1} \\ {} & = - K^{-1} (\partial_{p} K) K^{-1} \end{align}

given a square matrix K, we have

\begin{align} KK^{-1} &= I \\ \implies \frac{d(KK^{-1})}{dp} &=\frac{d(I)}{dp}\\ \end{align} for notational simplicity, we use $$dK = \frac{dK}{dp}$$, thus \begin{align} (dK)K^{-1} + K(dK^{-1})&=0\\ \implies K(dK^{-1})&= -(dK)K^{-1}\\ \implies dK^{-1}&= -K^{-1}(dK)K^{-1}\\ \end{align}

which yileds: $$\frac{dK^{-1}}{dp}= -K^{-1}\frac{dK}{dp}K^{-1}$$

Another related method is to use differentials. $$d\mathbf{K}^{-1}= -\mathbf{K}^{-1} (d\mathbf{K}) \mathbf{K}^{-1}$$ and $$d\mathbf{K}= \frac{\partial \mathbf{K}}{\partial p} dp$$ Thus $$d\mathbf{K}^{-1}= -\left[\mathbf{K}^{-1} \frac{\partial \mathbf{K}}{\partial p} \mathbf{K}^{-1} \right] dp$$ from which it follows that $$\frac{\partial \mathbf{K}^{-1}}{\partial p} = -\mathbf{K}^{-1} \frac{\partial \mathbf{K}}{\partial p} \mathbf{K}^{-1}$$