Suppose that I have a symmetric Toeplitz $n\times n$ matrix

$$\mathbf{A}=\left[\begin{array}{cccc}a_1&a_2&\cdots& a_n\\a_2&a_1&\cdots&a_{n-1}\\\vdots&\vdots&\ddots&\vdots\\a_n&a_{n-1}&\cdots&a_1\end{array}\right]$$

where $a_i \geq 0$, and a diagonal matrix

$$\mathbf{B}=\left[\begin{array}{cccc}b_1&0&\cdots& 0\\0&b_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&b_n\end{array}\right]$$

where $b_i = \frac{c}{\beta_i}$ for some constant $c>0$ such that $\beta_i>0$. Let


Can one express a partial derivative $\partial_{\beta_i} \operatorname{Tr}[\mathbf{M}]$ in closed form, where $\operatorname{Tr}[\mathbf{M}]$ is the trace operator?


2 Answers 2


Define some variables for convenience $$\eqalign{ P &= {\rm Diag}(\beta) \cr B &= cP^{-1} \cr b &= {\rm diag}(B) \cr S &= A+B \cr M &= AS^{-1}A \cr }$$ all of which are symmetric matrices, except for $b$ which is a vector.

Then the function and its differential can be expressed in terms of the Frobenius (:) product as $$\eqalign{ f &= {\rm tr}(M) \cr &= A^2 : S^{-1} \cr\cr df &= A^2 : dS^{-1} \cr &= -A^2 : S^{-1}\,dS\,S^{-1} \cr &= -S^{-1}A^2S^{-1} : dS \cr &= -S^{-1}A^2S^{-1} : dB \cr &= -S^{-1}A^2S^{-1} : c\,dP^{-1} \cr &= c\,S^{-1}A^2S^{-1} : P^{-1}\,dP\,P^{-1} \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : dP \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : {\rm Diag}(d\beta) \cr &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big)^T d\beta \cr }$$ So the derivative is $$\eqalign{ \frac{\partial f}{\partial\beta} &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big) \cr &= \frac{1}{c}\,{\rm diag}\big(BS^{-1}A^2S^{-1}B\big) \cr &= \Big(\frac{b\circ b}{c}\Big)\circ{\rm diag}\big(S^{-1}A^2S^{-1}\big) \cr\cr }$$ which uses Hadamard ($\circ$) products in the final expression. This is the same as joriki's result, but with more details.

  • $\begingroup$ The final expression can be written without Hadamard products as $$\large{ \frac{\partial f}{\partial\beta} = \frac{1}{c}B^2{\,\rm diag}\!\left(S^{-1}A^2S^{-1}\right) }$$ $\endgroup$
    – lynn
    Mar 5, 2023 at 20:26

Expanding $\mathbf A(\mathbf A + \mathbf B + \mathbf E)^{-1}\mathbf A$ in $\mathbf E$ yields $\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf A-\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf E(\mathbf A + \mathbf B)^{-1}\mathbf A$ up to first order. Thus

$$ \begin{eqnarray} \frac{\partial\operatorname{Tr}[M]}{\partial\beta_i} &=& -\operatorname{Tr}\left[\mathbf A(\mathbf A + \mathbf B)^{-1}\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\right] \\ &=& -\operatorname{Tr}\left[\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right] \\ &=& \frac c{\beta_i^2}\left((\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right)_{ii}\;. \end{eqnarray} $$

  • $\begingroup$ Thank you! I really appreciate the answer, but could you please elaborate on what $\mathbf{E}$ is? Is it an arbitrary matrix? Being unfamiliar with matrix calculus, I would really like to learn how one does something like this (parallels to vector calculus would be greatly appreciated)... Again, thank you! $\endgroup$
    – M.B.M.
    Mar 21, 2012 at 2:35
  • $\begingroup$ @Bullmoose: You're welcome. Perhaps I should have written $\Delta\mathbf B$ instead of $\mathbf E$. You can think of it as a Taylor expansion up to first order: expand $\mathbf B(\beta_i+\Delta\beta_i)$ as $\mathbf B(\beta_i+\Delta\beta_i)=\mathbf B(\beta_i)+\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, write $\Delta\mathbf B=\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, and expand the expression for small $\Delta\mathbf B$; then the linear term in $\Delta\mathbf B$ is the linear term $\Delta \operatorname{Tr}[M]=\Delta\beta_i(\partial\operatorname{Tr}[M]/\partial\beta_i)$. $\endgroup$
    – joriki
    Mar 21, 2012 at 2:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .