Linking to a question that I previously asked (Differentiate expressions involving symmetric matrix $\mathbf{D}=\mathrm{diag}(\tau)\Omega\mathrm{diag}(\tau)$ with respect to element of $\tau$), I now find that I need to find the value of the following:

$$ \frac{d(\mathbf{b}^{T}\mathbf{D}^{-1}\mathbf{b})}{db_{g}} $$

In the above, $\mathbf{D}^{-1}$ is the inverse of a symmetric $q$ by $q$ covariance matrix, and $\mathbf{b}$ is a vector of length $q$. I want to differentiate the expression with respect to the $g$th element of $\mathbf{b}$.

I can see (e.g. using equation 85 in matrix cookbook https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf how to find the value of the above if differentiating by $\mathbf{b}$, but cannot see a clear step-by-step explanation for a single element $b_g$. Any guidance / help / pointers would be greatly appreciated.

Background: This stems from manipulating a multivariate normal distribution for parameter $\mathbf{b}$ with mean vector equal to vector of zeros and covariance matrix equal to $\mathbf{D}$.


Take $f(b) = b^TD^{-1}b$. From the equation you mention, we have $$ \frac{\partial f}{\partial b} = 2D^{-1}b. $$ Note that this assumes that $D$ is symmetric (which applies in your case because $D$ is a covariance matrix) and that $D$ is independent of $b$.

Since we're taking a derivative with respect to a column-vector, extracting the individual partial derivative is simple. $\frac{\partial f}{\partial b_g}$ is just the $g$th entry of $\frac{\partial f}{\partial b}$. In particular: let $e_g$ denote the $g$th column of the size $q$ identity matrix. We have $$ \frac{\partial f}{\partial b} = e_g^T(2D^{-1}b) = 2(e_g^TD^{-1})b. $$ Note that $e_g^T D^{-1}$ is the $g$th row of $D^{-1}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.