Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose that I have a symmetric Toeplitz $n\times n$ matrix $\mathbf{A}=\left[\begin{array}{cccc}a_1&a_2&\cdots& a_n\\a_2&a_1&\cdots&a_{n-1}\\\vdots&\vdots&\ddots&\vdots\\a_n&a_{n-1}&\cdots&a_1\end{array}\right]$ such that $a_i\geq 0$, and a diagonal matrix $\mathbf{B}=\left[\begin{array}{cccc}b_1&0&\cdots& 0\\0&b_2&\cdots&0\\\vdots&\vdots&\ddots&\vdots\\0&0&\cdots&b_n\end{array}\right]$ where $b_i=c/\beta_i$ for some constant $c>0$ such that $\beta_i>0$.

Now let $\mathbf{M}=\mathbf{A}(\mathbf{A}+\mathbf{B})^{-1}\mathbf{A}$.

Can one express a partial derivative $\partial \operatorname{Tr}[\mathbf{M}]\bigg/{\partial\beta_i}$ in closed form, where $\operatorname{Tr}[\mathbf{M}]$ is the trace operator?

share|cite|improve this question
up vote 1 down vote accepted

Expanding $\mathbf A(\mathbf A + \mathbf B + \mathbf E)^{-1}\mathbf A$ in $\mathbf E$ yields $\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf A-\mathbf A(\mathbf A + \mathbf B)^{-1}\mathbf E(\mathbf A + \mathbf B)^{-1}\mathbf A$ up to first order. Thus

$$ \begin{eqnarray} \frac{\partial\operatorname{Tr}[M]}{\partial\beta_i} &=& -\operatorname{Tr}\left[\mathbf A(\mathbf A + \mathbf B)^{-1}\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\right] \\ &=& -\operatorname{Tr}\left[\frac{\partial\mathbf B}{\partial\beta_i}(\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right] \\ &=& \frac c{\beta_i^2}\left((\mathbf A + \mathbf B)^{-1}\mathbf A\mathbf A(\mathbf A + \mathbf B)^{-1}\right)_{ii}\;. \end{eqnarray} $$

share|cite|improve this answer
Thank you! I really appreciate the answer, but could you please elaborate on what $\mathbf{E}$ is? Is it an arbitrary matrix? Being unfamiliar with matrix calculus, I would really like to learn how one does something like this (parallels to vector calculus would be greatly appreciated)... Again, thank you! – M.B.M. Mar 21 '12 at 2:35
@Bullmoose: You're welcome. Perhaps I should have written $\Delta\mathbf B$ instead of $\mathbf E$. You can think of it as a Taylor expansion up to first order: expand $\mathbf B(\beta_i+\Delta\beta_i)$ as $\mathbf B(\beta_i+\Delta\beta_i)=\mathbf B(\beta_i)+\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, write $\Delta\mathbf B=\Delta\beta_i(\partial\mathbf B/\partial\beta_i)$, and expand the expression for small $\Delta\mathbf B$; then the linear term in $\Delta\mathbf B$ is the linear term $\Delta \operatorname{Tr}[M]=\Delta\beta_i(\partial\operatorname{Tr}[M]/\partial\beta_i)‌​$. – joriki Mar 21 '12 at 2:56

Define some variables for convenience $$\eqalign{ P &= {\rm Diag}(\beta) \cr B &= cP^{-1} \cr b &= {\rm diag}(B) \cr S &= A+B \cr M &= AS^{-1}A \cr }$$ all of which are symmetric matrices, except for $b$ which is a vector.

Then the function and its differential can be expressed in terms of the Frobenius (:) product as $$\eqalign{ f &= {\rm tr}(M) \cr &= A^2 : S^{-1} \cr\cr df &= A^2 : dS^{-1} \cr &= -A^2 : S^{-1}\,dS\,S^{-1} \cr &= -S^{-1}A^2S^{-1} : dS \cr &= -S^{-1}A^2S^{-1} : dB \cr &= -S^{-1}A^2S^{-1} : c\,dP^{-1} \cr &= c\,S^{-1}A^2S^{-1} : P^{-1}\,dP\,P^{-1} \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : dP \cr &= c\,P^{-1}S^{-1}A^2S^{-1}P^{-1} : {\rm Diag}(d\beta) \cr &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big)^T d\beta \cr }$$ So the derivative is $$\eqalign{ \frac{\partial f}{\partial\beta} &= {\rm diag}\big(c\,P^{-1}S^{-1}A^2S^{-1}P^{-1}\big) \cr &= \frac{1}{c}\,{\rm diag}\big(BS^{-1}A^2S^{-1}B\big) \cr &= \Big(\frac{b\circ b}{c}\Big)\circ{\rm diag}\big(S^{-1}A^2S^{-1}\big) \cr\cr }$$ which uses Hadamard ($\circ$) products in the final expression. This is the same as joriki's result, but with more details.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.