# Derivative of the von Neumann entropy

Related questions here and here

I'm trying to compute the derivative of $$S(X) = Tr(X\log(X))$$. An additional condition is that $$X = L^TL$$ for some $$L$$ and I want to take the derivative with respect to $$L$$. So essentially, I would like to find $$\frac{\partial Tr\left(L^TL\log(L^TL)\right)}{\partial L}$$ This is well defined as it's the derivative of a scalar function with respect to a matrix so I should obtain a matrix of size $$L$$.

My progress so far:

Use the product rule to break it up into two derivatives. So I have $$\frac{\partial Tr(L^T L C)}{\partial L}$$, where $$C = \log(L^TL)$$ but is treated as a constant. According to the matrix cookbook, I get $$L(C^T + C) = L(\log(L^TL))^T + L\log(L^TL)$$.

The other term, $$\frac{\partial Tr(C'\log(L^T L) )}{\partial L}$$, is more troublesome. I thought about letting $$L^TL = X$$ and use the chain rule, but I'm not sure how to deal with the $$\frac{\partial X}{\partial L}$$ term since I now have the derivative of a matrix with respect to a matrix (instead of scalar with matrix).

So the question, after my work, is now how to do take the derivative of the second term. Alternatively, if the product rule was a bad idea and I can do it more directly, that would also be very helpful. Thank you.

Consider the following scalar function and its derivative. \eqalign{ f(x) &= x\,\log(x) \cr f'(x) &= 1 + \log(x) } When the function is applied to a matrix argument the result is a matrix $$F = f(X)$$ The differential of the trace of $$F$$ is then \eqalign{ d\,{\rm Tr}(F) &= f'(X)^T:dX \cr &= \big(I+\log(X)\big)^T:dX \cr } Now assume that $$X$$ is described in terms of another matrix $$L$$, i.e. \eqalign{ X &= L^TL \cr dX &= L^T\,dL + dL^T\,L \cr } Substituting this into the previous result yields \eqalign{ d\,{\rm Tr}(F) &= \big(I+\log(L^TL)\big)^T:(L^T\,dL + dL^T\,L) \cr &= 2\,\big(I+\log(L^TL)\big):L^T\,dL \cr &= 2\,L\big(I+\log(L^TL)\big):dL \cr \frac{\partial{\,\rm Tr}(F)}{\partial L} &= 2\,\big(L+L\log(L^TL)\big) \cr\cr } NB: In some steps above, a colon is used as a product notation for the trace, i.e. $$A:B = {\rm Tr}(A^TB)$$
• Great answer! Just one question: How did you go from $\big(I+\log(L^TL)\big)^T:(L^T\,dL + dL^T\,L)$ to $2\big(I+\log(L^TL)\big):(L^T\,dL)$? Thank you – user1936752 Feb 23 at 20:49
• It follows from these easily proved statements \eqalign{ (L^TdL)^T &= (L^TdL) \cr (\log(L^TL)+I)^T &= (\log(L^TL)+I) \cr A^T:B &= A:B^T \cr } – greg Feb 23 at 23:12