# Finding the derivative of inverse of the product of matrices

I need to calculate the following derivative of the product of several matrices (one of which is the inverse of a product of matrices) with respect to one of the matrices in question:

$$\frac{\delta(\mathbf{a}^T(\text{diag}(\mathbf{\tau}) \mathbf{\Omega} \text{diag}(\mathbf{\tau}))^{-1}\mathbf{a})}{\delta\mathbf{\Omega}}$$

In the above, $$\mathbf{a}$$ and $$\mathbf{\tau}$$ are a vectors, $$\text{diag}(\mathbf{\tau})$$ is the diagonal matrix with elements of $$\mathbf{\tau}$$ on its diagonal, and $$\mathbf{\Omega}$$ is a symmetric matrix. The elements in vectors $$\mathbf{a}$$ and $$\mathbf{\tau}$$ don't rely on any element of $$\mathbf{\Omega}$$.

From the matrix cookbook, I can see rules such as

$$\frac{\delta\mathbf{a}^T\mathbf{X}^{-1}\mathbf{b}}{\delta\mathbf{X}}=-\mathbf{X}^{-T}\mathbf{ab}^T\mathbf{X}^{-T}$$

However, I'm having trouble finding a rule for when you are differentiating by a matrix which is only one of a product of matrices that is being inverted? Any guidance would be appreciated.

(Context - I am trying to calculate values for the derivatives of the log posterior of a model with respect to different parameters (in this case the correlation matrix $$\mathbf{\Omega}$$) in order to write a model fitting algorithm)

## 1 Answer

Define the matrices $${ B = B^T = {\rm Diag}(\tau), \quad X = X^T = B\,\Omega\,B, \quad dX = B\,d\Omega\,B }$$ Use the result from the cookbook (with $$b=a$$) to write the differential of the function (in terms of $$dX$$), then substitute the above definitions (in terms of $$d\Omega$$) and find the gradient (with respect to $$\Omega$$). \eqalign{ \phi &= a^TX^{-1}a \cr d\phi &= \Big(\frac{\partial\phi}{\partial X}\Big):dX = -(X^{-T}aa^TX^{-T}):dX \cr &= -(X^{-1}aa^TX^{-1}):(B\,d\Omega\,B) \cr &= -(BX^{-1}aa^TX^{-1}B):d\Omega \cr &= -\big(\Omega^{-1}B^{-1}aa^TB^{-1}\Omega^{-1}\big):d\Omega \cr \frac{\partial\phi}{\partial\Omega} &= -\Omega^{-1}B^{-1}aa^TB^{-1}\Omega^{-1} \cr } where a colon has been used to denote the trace/Frobenius product, i.e. $$\,A:B = {\rm Tr}(A^TB)$$