# Derivative of a trace with respect to a matrix

Could someone explain this equation?

$$\frac{d \operatorname{tr}(AXB)}{d X} = BA$$

I understand that

$$d\operatorname{tr}(AXB) = \operatorname{tr}(BA \; dX)$$

but I don't quite understand how to move $dX$ out of the trace.

The notation is quite misleading (at least for me).

Hint:

Does it make sense that $$\frac{\partial}{\partial X_{mn}} \mathop{\rm tr} (A X B) = (B A)_{nm}?$$

More information: $$\frac{\partial}{\partial X_{mn}} \mathop{\rm tr} (A X B) = \frac{\partial}{\partial X_{mn}} \sum_{jkl} A_{jk} X_{kl} B_{lj} = \sum_{jkl} A_{jk} \delta_{km} \delta_{nl} B_{lj} = \sum_{j} A_{jm} B_{nj} =(B A)_{nm}.$$

• That does make sense if I assume the first equation of $\frac{dtr(AXB)}{dX}$. But I'm not sure how to get to the first equation using the second equation for $dtr(AXB)$. Mar 11, 2012 at 5:37
• @ChrisD: I added a line explaining how to use my hint. Mar 11, 2012 at 12:41
• I see what you mean by misleading notation. The resulting matrix is indexed by the transpose of the matrix you differentiate by, $X$. Therefore, the full matrix solution $BA$ is the transpose of the element by element solution $(BA)_{nm}$. Thanks for the help. Mar 11, 2012 at 20:10
• @Fabian is there any book that covers it? the Matrix Cookbook does not show the derivations but only the results... so I prefer something else Mar 11, 2020 at 10:12
• I think this is the right notation because the point is that the "derivative with respect to a matrix" is just the usual gradient of a scalar function... which in this very particular case happens to be easily reshaped as a matrix with the same dimensions as $X$. Oct 16, 2021 at 11:14

These are the main equations to remember:

1. Let $$\mathbf{A} \in \mathbb{R}^{n\times m}$$, $$\mathbf{X} \in \mathbb{R}^{m\times n}$$. Then

$$$$\frac{d}{d\mathbf{X}}\text{Tr}(\mathbf{AX}) = \frac{d}{d\mathbf{X}}\text{Tr}(\mathbf{XA}) = \mathbf{A}^T$$$$

1. Let $$\mathbf{A} \in \mathbb{R}^{n\times m}$$, $$\mathbf{X} \in \mathbb{R}^{n\times m}$$. Then

$$$$\frac{d}{d\mathbf{X}}\text{Tr}(\mathbf{AX^T}) = \frac{d}{d\mathbf{X}}\text{Tr}(\mathbf{X^TA}) = \mathbf{A}$$$$

Proof 1.

$$$$\left[ \frac{d}{d\mathbf{X}} \text{Tr}(\mathbf{AX}) \right]_{i,j} = \frac{d}{dx_{i,j}} \text{Tr}(\mathbf{AX}) = \frac{d}{dx_{i,j}} \sum_{k,l} a_{k,l} x_{l,k} = a_{j,i} = \left[\mathbf{A}^T\right]_{i,j}$$$$

Proof 2

$$$$\left[ \frac{d}{d\mathbf{X}} \text{Tr}(\mathbf{AX^T}) \right]_{i,j} = \frac{d}{dx_{i,j}} \text{Tr}(\mathbf{AX^T}) = \frac{d}{dx_{i,j}} \sum_{k,l} a_{k,l} x_{k,l} = a_{i,j} = \left[\mathbf{A}\right]_{i,j}$$$$

Once you have these, you can derivate crazy things like the following:

Example 1. Let $$\mathbf{A} \in \mathbb{R}^{m\times m}$$, $$\mathbf{X} \in \mathbb{R}^{m\times n}$$. Then

$$$$\frac{d}{d\mathbf{X}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{X}) = (\mathbf{A} + \mathbf{A}^T) \mathbf{X}$$$$

We can derive it as follows:

$$\begin{split} \frac{d}{d\mathbf{X}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{X}) =& \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{Y}^T \mathbf{A} \mathbf{X}) + \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{Y})\\ =& \mathbf{A} \mathbf{X} + (\mathbf{X}^T \mathbf{A})^T\\ =& \mathbf{A} \mathbf{X} + \mathbf{A}^T \mathbf{X} = (\mathbf{A} + \mathbf{A}^T) \mathbf{X} \end{split}$$

Example 2. Consider now this example.

$$$$f(\mathbf{X}) = \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{X}^T \mathbf{C})$$$$

where $$\mathbf{X} \in \mathbb{R}^{n\times m}$$, $$\mathbf{A} \in \mathbb{R}^{n\times n}$$, $$\mathbf{B} \in \mathbb{R}^{m\times m}$$, $$\mathbf{C} \in \mathbb{R}^{n\times m}$$.

$$$$\begin{split} \frac{d}{d\mathbf{X}} f(\mathbf{X}) =& \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{Y}^T \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{X}^T \mathbf{C})\\ +& \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{Y} \mathbf{B} \mathbf{X}^T \mathbf{C})\\ +& \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{Y}^T \mathbf{C}) \end{split}$$$$

Calculating these:

$$\begin{split} \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{Y}^T \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{X}^T \mathbf{C}) = \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{X}^T \mathbf{C} \end{split}$$

$$\begin{split} \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{Y} \mathbf{B} \mathbf{X}^T \mathbf{C}) =& \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{Y} \mathbf{B} \mathbf{X}^T \mathbf{C} \mathbf{X}^T \mathbf{A})\\ =& (\mathbf{B} \mathbf{X}^T \mathbf{C} \mathbf{X}^T \mathbf{A})^T\\ =& \mathbf{A}^T \mathbf{X} \mathbf{C}^T \mathbf{X} \mathbf{B}^T \end{split}$$

$$\begin{split} \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{Y}^T \mathbf{C}) = \frac{d}{d\mathbf{Y}} \text{Tr}(\mathbf{Y}^T \mathbf{C} \mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B}) = \mathbf{C} \mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B} \end{split}$$

So the result is:

$$$$\frac{d}{d\mathbf{X}} \text{Tr}(\mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{X}^T \mathbf{C}) = \mathbf{A} \mathbf{X} \mathbf{B} \mathbf{X}^T \mathbf{C} + \mathbf{A}^T \mathbf{X} \mathbf{C}^T \mathbf{X} \mathbf{B}^T + \mathbf{C} \mathbf{X}^T \mathbf{A} \mathbf{X} \mathbf{B}$$$$

Try expanding to linear order. This always eases the understanding:

$$\operatorname{tr}(A (X+dX)B)=A_{ij} (X_{jk}+dX_{jk})B_{ki}$$

where Einstein's summation rule is used. Substracting $$\operatorname{tr}(AXB)$$ you get

\begin{align} d\operatorname{tr}(AXB)&=\operatorname{tr}(A(X+dX)B)-\operatorname{tr}(AXB)\\&=A_{ij} dX_{jk}B_{ki}=\underbrace{B_{ki}A_{ij}}_{=(BA)_{kj}} \; dX_{jk} \end{align}

• When you say 'expanding to linear order', do you mean write out the actual matrix element summations? I had to do that manually to prove $$\operatorname{tr}(A (X+dX)B)=A_{ij} (X_{jk}+dX_{jk})B_{ki})$$ to myself. Is that something that's easily derived without explicitly expanding the matrices or is this something people generally just memorize about traces? Mar 11, 2012 at 20:32
• What I meant was writing $X+dX$ and work with that, keeping only stuff that is linear in $dX$. In this case, everything was linear to begin with, so my comment was a bit misleading. But it is the right way to work when deriving more complicated tensorial derivatives like, say, $$\frac{\partial \det (A)}{\partial A}=\det(A) \left(A^{-1}\right)^T$$ Mar 12, 2012 at 5:35

The other answers are correct, but I feel like they missed the point. Arguments that take a basis to prove a result independent of bases should be approached with caution.

First of all, according to the Matrix Cookbook, the formula is $$\frac{d\mathrm{tr}(AXB)}{dX} = (BA)^T,$$ not the one given in your question.

What's confusing about this presentation is that $$f (X) = \mathrm{tr}(AXB)$$ is a linear map, so it's derivative (=linear approximation) is itself.

So in fact, the statement should read $$f(X) = \mathrm{tr}(AXB) = (BA)^T,$$ which is clearly wrong.

But consider the Frobenius inner product on $$\mathrm{Mat}(m, n)$$. For $$U, V \in \mathrm{Mat}(m, n)$$:

$$\langle U, V \rangle = \mathrm{tr}(U^T V).$$

By the Riesz representation theorem, $$f$$ can be represented as

$$f(X) = \langle U, X \rangle = \mathrm{tr}(U^TX).$$

for a fixed $$U \in \mathrm{Mat}(m, n)$$.

Clearly $$U = (BA)^T$$ does the job, so the more precise statement is

$$\mathrm{tr}(AXB) = \langle (BA)^T, X \rangle,$$

which is a triviality.