# Inverse Matrix Differential

Suppose that we are given $$g(\Sigma)=\Sigma^{-1}(\mu_0-\mu_1)$$, where $$\Sigma$$ is p by p, and both $$\mu_0$$ and $$\mu_1$$ are p by 1.

Now I am hoping to find $$\frac{dg}{d\Sigma}$$. My current work is using the idea of differential:

$$dg = (d\Sigma^{-1})(\mu_0-\mu_1) = (-\Sigma^{-1}(d\Sigma)\Sigma^{-1})(\mu_0-\mu_1)$$

and I am stuck.

I would really appreciate it if you may kindly show me the working steps.

p.s.:I believe the final result should be of dimension $$\frac{1}{2}p(p+1)$$ by $$p$$.

You are attempting to find the derivative (gradient) of a vector with respect to a matrix. The result will not be a matrix, but a 3rd order tensor!

One way to proceed is to use vectorization (aka column stacking).

And for ease of typing define: $$\,\,S=\Sigma,\,\,\,W=S^{-1},\,\,\,s={\rm vec}(S),\,\,\,y=(\mu_1-\mu_0)$$

Since your differential is correct, let's start from there. \eqalign{ {\rm vec}(dg) &= {\rm vec}\Big(W\,dS\,Wy\Big) \cr dg &= \Big((Wy)^T\otimes W\Big)\,ds \cr \frac{\partial g}{\partial s} &= (Wy)^T\otimes W \cr } Another way to proceed is to use higher-order tensors, or else index notation.

In latter notation, the 3rd-order tensor gradient can be written as \eqalign{ G_{ijk} &= \frac{\partial g_{i}}{\partial S_{jk}} = W_{ik}W_{jn}y_{n} \cr } where the repeated index $$(n)$$ in the final terms implies a summation over that index.

• Yes! This is what I am looking for. Thank you! If possible, could you please kindly tell me where I may obtain more information about derivatives with respect to matrices (like advanced linear algebra?)? – NeverBe Oct 19 '18 at 4:45
• If you want to learn about vectorization, try Magnus & Neudecker. If you're interested in index notation, look for a good textbook in the field of Continuum Mechanics. – greg Oct 19 '18 at 12:30
• Sounds good. Thank you! – NeverBe Oct 19 '18 at 19:31