Partial derivative involving trace of a matrix Suppose that I have a symmetric Toeplitz $n\times n$ matrix
$$\mathbf{A}=\left[\begin{array}{cccc}a_1&a_2&\cdots& a_n\\a_2&a_1&\cdots&a_{n-1}\\\vdots&\vdots&\ddots&\vdots\\a_n&a_{n-1}&\cdots&a_1\end{array}\right]$$
where $a_i \geq 0$, and a diagonal matrix
$$\mathbf{B}=\left[\begin{array}{cccc}b_1&0&\cdots& 0\\0&b_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&b_n\end{array}\right]$$
where $b_i = \frac{c}{\beta_i}$ for some constant $c>0$ such that $\beta_i>0$. Let
$$\mathbf{M}=\mathbf{A}(\mathbf{A}+\mathbf{B})^{-1}\mathbf{A}$$
Can one express a partial derivative $\partial_{\beta_i} \operatorname{Tr}[\mathbf{M}]$ in closed form, where $\operatorname{Tr}[\mathbf{M}]$ is the trace operator?
 A: Define some variables for convenience 
$$\eqalign{
  P &= {\rm Diag}(\beta) \cr
  B &= cP^{-1} \cr
  b &= {\rm diag}(B) \cr
  S &= A+B \cr
  M &= AS^{-1}A \cr
}$$
all of which are symmetric matrices, except for $b$ which is a vector.
Then the function and its differential can be expressed in terms of the Frobenius (:) product as
$$\eqalign{
  f &= {\rm tr}(M) \cr
    &= A^2 : S^{-1} \cr\cr
  df &= A^2 : dS^{-1} \cr
     &= -A^2 : S^{-1}\,dS\,S^{-1} \cr
     &= -S^{-1}A^2S^{-1} : dS \cr
     &= -S^{-1}A^2S^{-1} : dB \cr
     &= -S^{-1}A^2S^{-1} : c\,dP^{-1} \cr
     &= c\,S^{-1}A^2S^{-1} : P^{-1}\,dP\,P^{-1} \cr
     &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : dP \cr
     &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : {\rm Diag}(d\beta) \cr
     &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big)^T d\beta \cr
}$$
So the derivative is
$$\eqalign{
 \frac{\partial f}{\partial\beta} &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big) \cr
  &= \frac{1}{c}\,{\rm diag}\big(BS^{-1}A^2S^{-1}B\big) \cr
  &= \Big(\frac{b\circ b}{c}\Big)\circ{\rm diag}\big(S^{-1}A^2S^{-1}\big) \cr\cr
}$$
which uses Hadamard ($\circ$) products in the final expression. This is the same as joriki's result, but with more details.
A: Expanding $\mathbf A(\mathbf A + \mathbf B + \mathbf E)^{-1}\mathbf A$ in $\mathbf E$ yields $\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf A-\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf E(\mathbf A + \mathbf B)^{-1}\mathbf A$ up to first order. Thus
$$
\begin{eqnarray}
\frac{\partial\operatorname{Tr}[M]}{\partial\beta_i}
&=&
-\operatorname{Tr}\left[\mathbf A(\mathbf A + \mathbf B)^{-1}\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\right]
\\
&=&
-\operatorname{Tr}\left[\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right]
\\
&=&
\frac c{\beta_i^2}\left((\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right)_{ii}\;.
\end{eqnarray}
$$
