# Partial derivative involving trace of a matrix

Suppose that I have a symmetric Toeplitz $n\times n$ matrix $\mathbf{A}=\left[\begin{array}{cccc}a_1&a_2&\cdots& a_n\\a_2&a_1&\cdots&a_{n-1}\\\vdots&\vdots&\ddots&\vdots\\a_n&a_{n-1}&\cdots&a_1\end{array}\right]$ such that $a_i\geq 0$, and a diagonal matrix $\mathbf{B}=\left[\begin{array}{cccc}b_1&0&\cdots& 0\\0&b_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&b_n\end{array}\right]$ where $b_i=c/\beta_i$ for some constant $c>0$ such that $\beta_i>0$.

Now let $\mathbf{M}=\mathbf{A}(\mathbf{A}+\mathbf{B})^{-1}\mathbf{A}$.

Can one express a partial derivative $\partial \operatorname{Tr}[\mathbf{M}]\bigg/{\partial\beta_i}$ in closed form, where $\operatorname{Tr}[\mathbf{M}]$ is the trace operator?

-

Expanding $\mathbf A(\mathbf A + \mathbf B + \mathbf E)^{-1}\mathbf A$ in $\mathbf E$ yields $\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf A-\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf E(\mathbf A + \mathbf B)^{-1}\mathbf A$ up to first order. Thus

$$\begin{eqnarray} \frac{\partial\operatorname{Tr}[M]}{\partial\beta_i} &=& -\operatorname{Tr}\left[\mathbf A(\mathbf A + \mathbf B)^{-1}\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\right] \\ &=& -\operatorname{Tr}\left[\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right] \\ &=& \frac c{\beta_i^2}\left((\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right)_{ii}\;. \end{eqnarray}$$

-
Thank you! I really appreciate the answer, but could you please elaborate on what $\mathbf{E}$ is? Is it an arbitrary matrix? Being unfamiliar with matrix calculus, I would really like to learn how one does something like this (parallels to vector calculus would be greatly appreciated)... Again, thank you! – M.B.M. Mar 21 '12 at 2:35
@Bullmoose: You're welcome. Perhaps I should have written $\Delta\mathbf B$ instead of $\mathbf E$. You can think of it as a Taylor expansion up to first order: expand $\mathbf B(\beta_i+\Delta\beta_i)$ as $\mathbf B(\beta_i+\Delta\beta_i)=\mathbf B(\beta_i)+\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, write $\Delta\mathbf B=\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, and expand the expression for small $\Delta\mathbf B$; then the linear term in $\Delta\mathbf B$ is the linear term $\Delta \operatorname{Tr}[M]=\Delta\beta_i(\partial\operatorname{Tr}[M]/\partial\beta_i)‌​$. – joriki Mar 21 '12 at 2:56

Define some variables for convenience \eqalign{ P &= {\rm Diag}(\beta) \cr B &= cP^{-1} \cr b &= {\rm diag}(B) \cr S &= A+B \cr M &= AS^{-1}A \cr } all of which are symmetric matrices, except for $b$ which is a vector.

Then the function and its differential can be expressed in terms of the Frobenius (:) product as \eqalign{ f &= {\rm tr}(M) \cr &= A^2 : S^{-1} \cr\cr df &= A^2 : dS^{-1} \cr &= -A^2 : S^{-1}\,dS\,S^{-1} \cr &= -S^{-1}A^2S^{-1} : dS \cr &= -S^{-1}A^2S^{-1} : dB \cr &= -S^{-1}A^2S^{-1} : c\,dP^{-1} \cr &= c\,S^{-1}A^2S^{-1} : P^{-1}\,dP\,P^{-1} \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : dP \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : {\rm Diag}(d\beta) \cr &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big)^T d\beta \cr } So the derivative is \eqalign{ \frac{\partial f}{\partial\beta} &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big) \cr &= \frac{1}{c}\,{\rm diag}\big(BS^{-1}A^2S^{-1}B\big) \cr &= \Big(\frac{b\circ b}{c}\Big)\circ{\rm diag}\big(S^{-1}A^2S^{-1}\big) \cr\cr } which uses Hadamard ($\circ$) products in the final expression. This is the same as joriki's result, but with more details.

-