Suppose that I have a symmetric Toeplitz $n\times n$ matrix

$$\mathbf{A}=\left[\begin{array}{cccc}a_1&a_2&\cdots& a_n\\a_2&a_1&\cdots&a_{n-1}\\\vdots&\vdots&\ddots&\vdots\\a_n&a_{n-1}&\cdots&a_1\end{array}\right]$$

where $a_i \geq 0$, and a diagonal matrix

$$\mathbf{B}=\left[\begin{array}{cccc}b_1&0&\cdots& 0\\0&b_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&b_n\end{array}\right]$$

where $b_i = \frac{c}{\beta_i}$ for some constant $c>0$ such that $\beta_i>0$. Let


Can one express a partial derivative $\partial_{\beta_i} \operatorname{Tr}[\mathbf{M}]$ in closed form, where $\operatorname{Tr}[\mathbf{M}]$ is the trace operator?


Define some variables for convenience $$\eqalign{ P &= {\rm Diag}(\beta) \cr B &= cP^{-1} \cr b &= {\rm diag}(B) \cr S &= A+B \cr M &= AS^{-1}A \cr }$$ all of which are symmetric matrices, except for $b$ which is a vector.

Then the function and its differential can be expressed in terms of the Frobenius (:) product as $$\eqalign{ f &= {\rm tr}(M) \cr &= A^2 : S^{-1} \cr\cr df &= A^2 : dS^{-1} \cr &= -A^2 : S^{-1}\,dS\,S^{-1} \cr &= -S^{-1}A^2S^{-1} : dS \cr &= -S^{-1}A^2S^{-1} : dB \cr &= -S^{-1}A^2S^{-1} : c\,dP^{-1} \cr &= c\,S^{-1}A^2S^{-1} : P^{-1}\,dP\,P^{-1} \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : dP \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : {\rm Diag}(d\beta) \cr &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big)^T d\beta \cr }$$ So the derivative is $$\eqalign{ \frac{\partial f}{\partial\beta} &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big) \cr &= \frac{1}{c}\,{\rm diag}\big(BS^{-1}A^2S^{-1}B\big) \cr &= \Big(\frac{b\circ b}{c}\Big)\circ{\rm diag}\big(S^{-1}A^2S^{-1}\big) \cr\cr }$$ which uses Hadamard ($\circ$) products in the final expression. This is the same as joriki's result, but with more details.


Expanding $\mathbf A(\mathbf A + \mathbf B + \mathbf E)^{-1}\mathbf A$ in $\mathbf E$ yields $\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf A-\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf E(\mathbf A + \mathbf B)^{-1}\mathbf A$ up to first order. Thus

$$ \begin{eqnarray} \frac{\partial\operatorname{Tr}[M]}{\partial\beta_i} &=& -\operatorname{Tr}\left[\mathbf A(\mathbf A + \mathbf B)^{-1}\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\right] \\ &=& -\operatorname{Tr}\left[\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right] \\ &=& \frac c{\beta_i^2}\left((\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right)_{ii}\;. \end{eqnarray} $$

  • $\begingroup$ Thank you! I really appreciate the answer, but could you please elaborate on what $\mathbf{E}$ is? Is it an arbitrary matrix? Being unfamiliar with matrix calculus, I would really like to learn how one does something like this (parallels to vector calculus would be greatly appreciated)... Again, thank you! $\endgroup$
    – M.B.M.
    Mar 21 '12 at 2:35
  • $\begingroup$ @Bullmoose: You're welcome. Perhaps I should have written $\Delta\mathbf B$ instead of $\mathbf E$. You can think of it as a Taylor expansion up to first order: expand $\mathbf B(\beta_i+\Delta\beta_i)$ as $\mathbf B(\beta_i+\Delta\beta_i)=\mathbf B(\beta_i)+\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, write $\Delta\mathbf B=\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, and expand the expression for small $\Delta\mathbf B$; then the linear term in $\Delta\mathbf B$ is the linear term $\Delta \operatorname{Tr}[M]=\Delta\beta_i(\partial\operatorname{Tr}[M]/\partial\beta_i)$. $\endgroup$
    – joriki
    Mar 21 '12 at 2:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.