I need to calculate the following derivative of the product of several matrices (one of which is the inverse of a product of matrices) with respect to one of the matrices in question:

$$\frac{\delta(\mathbf{a}^T(\text{diag}(\mathbf{\tau}) \mathbf{\Omega} \text{diag}(\mathbf{\tau}))^{-1}\mathbf{a})}{\delta\mathbf{\Omega}}$$

In the above, $\mathbf{a}$ and $\mathbf{\tau}$ are a vectors, $\text{diag}(\mathbf{\tau})$ is the diagonal matrix with elements of $\mathbf{\tau}$ on its diagonal, and $\mathbf{\Omega}$ is a symmetric matrix. The elements in vectors $\mathbf{a}$ and $\mathbf{\tau}$ don't rely on any element of $\mathbf{\Omega}$.

From the matrix cookbook, I can see rules such as


However, I'm having trouble finding a rule for when you are differentiating by a matrix which is only one of a product of matrices that is being inverted? Any guidance would be appreciated.

(Context - I am trying to calculate values for the derivatives of the log posterior of a model with respect to different parameters (in this case the correlation matrix $\mathbf{\Omega}$) in order to write a model fitting algorithm)


Define the matrices $${ B = B^T = {\rm Diag}(\tau), \quad X = X^T = B\,\Omega\,B, \quad dX = B\,d\Omega\,B }$$ Use the result from the cookbook (with $b=a$) to write the differential of the function (in terms of $dX$), then substitute the above definitions (in terms of $d\Omega$) and find the gradient (with respect to $\Omega$). $$\eqalign{ \phi &= a^TX^{-1}a \cr d\phi &= \Big(\frac{\partial\phi}{\partial X}\Big):dX = -(X^{-T}aa^TX^{-T}):dX \cr &= -(X^{-1}aa^TX^{-1}):(B\,d\Omega\,B) \cr &= -(BX^{-1}aa^TX^{-1}B):d\Omega \cr &= -\big(\Omega^{-1}B^{-1}aa^TB^{-1}\Omega^{-1}\big):d\Omega \cr \frac{\partial\phi}{\partial\Omega} &= -\Omega^{-1}B^{-1}aa^TB^{-1}\Omega^{-1} \cr }$$ where a colon has been used to denote the trace/Frobenius product, i.e. $\,A:B = {\rm Tr}(A^TB)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.