Suppose that we are given $g(\Sigma)=\Sigma^{-1}(\mu_0-\mu_1)$, where $\Sigma$ is p by p, and both $\mu_0$ and $\mu_1$ are p by 1.

Now I am hoping to find $\frac{dg}{d\Sigma}$. My current work is using the idea of differential:

$$ dg = (d\Sigma^{-1})(\mu_0-\mu_1) = (-\Sigma^{-1}(d\Sigma)\Sigma^{-1})(\mu_0-\mu_1) $$

and I am stuck.

I would really appreciate it if you may kindly show me the working steps.

p.s.:I believe the final result should be of dimension $\frac{1}{2}p(p+1)$ by $p$.


You are attempting to find the derivative (gradient) of a vector with respect to a matrix. The result will not be a matrix, but a 3rd order tensor!

One way to proceed is to use vectorization (aka column stacking).

And for ease of typing define: $\,\,S=\Sigma,\,\,\,W=S^{-1},\,\,\,s={\rm vec}(S),\,\,\,y=(\mu_1-\mu_0)$

Since your differential is correct, let's start from there. $$\eqalign{ {\rm vec}(dg) &= {\rm vec}\Big(W\,dS\,Wy\Big) \cr dg &= \Big((Wy)^T\otimes W\Big)\,ds \cr \frac{\partial g}{\partial s} &= (Wy)^T\otimes W \cr }$$ Another way to proceed is to use higher-order tensors, or else index notation.

In latter notation, the 3rd-order tensor gradient can be written as $$\eqalign{ G_{ijk} &= \frac{\partial g_{i}}{\partial S_{jk}} = W_{ik}W_{jn}y_{n} \cr }$$ where the repeated index $(n)$ in the final terms implies a summation over that index.

  • $\begingroup$ Yes! This is what I am looking for. Thank you! If possible, could you please kindly tell me where I may obtain more information about derivatives with respect to matrices (like advanced linear algebra?)? $\endgroup$ – NeverBe Oct 19 '18 at 4:45
  • 1
    $\begingroup$ If you want to learn about vectorization, try Magnus & Neudecker. If you're interested in index notation, look for a good textbook in the field of Continuum Mechanics. $\endgroup$ – greg Oct 19 '18 at 12:30
  • $\begingroup$ Sounds good. Thank you! $\endgroup$ – NeverBe Oct 19 '18 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.