# Partial derivative involving trace of a matrix

Suppose that I have a symmetric Toeplitz $$n\times n$$ matrix

$$\mathbf{A}=\left[\begin{array}{cccc}a_1&a_2&\cdots& a_n\\a_2&a_1&\cdots&a_{n-1}\\\vdots&\vdots&\ddots&\vdots\\a_n&a_{n-1}&\cdots&a_1\end{array}\right]$$

where $$a_i \geq 0$$, and a diagonal matrix

$$\mathbf{B}=\left[\begin{array}{cccc}b_1&0&\cdots& 0\\0&b_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&b_n\end{array}\right]$$

where $$b_i = \frac{c}{\beta_i}$$ for some constant $$c>0$$ such that $$\beta_i>0$$. Let

$$\mathbf{M}=\mathbf{A}(\mathbf{A}+\mathbf{B})^{-1}\mathbf{A}$$

Can one express a partial derivative $$\partial_{\beta_i} \operatorname{Tr}[\mathbf{M}]$$ in closed form, where $$\operatorname{Tr}[\mathbf{M}]$$ is the trace operator?

Define some variables for convenience \eqalign{ P &= {\rm Diag}(\beta) \cr B &= cP^{-1} \cr b &= {\rm diag}(B) \cr S &= A+B \cr M &= AS^{-1}A \cr } all of which are symmetric matrices, except for $b$ which is a vector.

Then the function and its differential can be expressed in terms of the Frobenius (:) product as \eqalign{ f &= {\rm tr}(M) \cr &= A^2 : S^{-1} \cr\cr df &= A^2 : dS^{-1} \cr &= -A^2 : S^{-1}\,dS\,S^{-1} \cr &= -S^{-1}A^2S^{-1} : dS \cr &= -S^{-1}A^2S^{-1} : dB \cr &= -S^{-1}A^2S^{-1} : c\,dP^{-1} \cr &= c\,S^{-1}A^2S^{-1} : P^{-1}\,dP\,P^{-1} \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : dP \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : {\rm Diag}(d\beta) \cr &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big)^T d\beta \cr } So the derivative is \eqalign{ \frac{\partial f}{\partial\beta} &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big) \cr &= \frac{1}{c}\,{\rm diag}\big(BS^{-1}A^2S^{-1}B\big) \cr &= \Big(\frac{b\circ b}{c}\Big)\circ{\rm diag}\big(S^{-1}A^2S^{-1}\big) \cr\cr } which uses Hadamard ($\circ$) products in the final expression. This is the same as joriki's result, but with more details.

• The final expression can be written without Hadamard products as $$\large{ \frac{\partial f}{\partial\beta} = \frac{1}{c}B^2{\,\rm diag}\!\left(S^{-1}A^2S^{-1}\right) }$$
– lynn
Mar 5, 2023 at 20:26

Expanding $\mathbf A(\mathbf A + \mathbf B + \mathbf E)^{-1}\mathbf A$ in $\mathbf E$ yields $\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf A-\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf E(\mathbf A + \mathbf B)^{-1}\mathbf A$ up to first order. Thus

$$\begin{eqnarray} \frac{\partial\operatorname{Tr}[M]}{\partial\beta_i} &=& -\operatorname{Tr}\left[\mathbf A(\mathbf A + \mathbf B)^{-1}\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\right] \\ &=& -\operatorname{Tr}\left[\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right] \\ &=& \frac c{\beta_i^2}\left((\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right)_{ii}\;. \end{eqnarray}$$

• Thank you! I really appreciate the answer, but could you please elaborate on what $\mathbf{E}$ is? Is it an arbitrary matrix? Being unfamiliar with matrix calculus, I would really like to learn how one does something like this (parallels to vector calculus would be greatly appreciated)... Again, thank you! Mar 21, 2012 at 2:35
• @Bullmoose: You're welcome. Perhaps I should have written $\Delta\mathbf B$ instead of $\mathbf E$. You can think of it as a Taylor expansion up to first order: expand $\mathbf B(\beta_i+\Delta\beta_i)$ as $\mathbf B(\beta_i+\Delta\beta_i)=\mathbf B(\beta_i)+\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, write $\Delta\mathbf B=\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, and expand the expression for small $\Delta\mathbf B$; then the linear term in $\Delta\mathbf B$ is the linear term $\Delta \operatorname{Tr}[M]=\Delta\beta_i(\partial\operatorname{Tr}[M]/\partial\beta_i)$. Mar 21, 2012 at 2:56