Derivative of the von Neumann entropy

Question

Related questions here and here

I'm trying to compute the derivative of $S(X) = Tr(X\log(X))$. An additional condition is that $X = L^TL$ for some $L$ and I want to take the derivative with respect to $L$. So essentially, I would like to find $$\frac{\partial Tr\left(L^TL\log(L^TL)\right)}{\partial L}$$ This is well defined as it's the derivative of a scalar function with respect to a matrix so I should obtain a matrix of size $L$.

My progress so far:

Use the product rule to break it up into two derivatives. So I have $\frac{\partial Tr(L^T L C)}{\partial L}$, where $C = \log(L^TL)$ but is treated as a constant. According to the matrix cookbook, I get $L(C^T + C) = L(\log(L^TL))^T + L\log(L^TL)$.

The other term, $\frac{\partial Tr(C'\log(L^T L) )}{\partial L}$, is more troublesome. I thought about letting $L^TL = X$ and use the chain rule, but I'm not sure how to deal with the $\frac{\partial X}{\partial L}$ term since I now have the derivative of a matrix with respect to a matrix (instead of scalar with matrix).

So the question, after my work, is now how to do take the derivative of the second term. Alternatively, if the product rule was a bad idea and I can do it more directly, that would also be very helpful. Thank you.

Answer 1

Consider the following scalar function and its derivative. $$\eqalign{ f(x) &= x\,\log(x) \cr f'(x) &= 1 + \log(x) }$$ When the function is applied to a matrix argument the result is a matrix $$F = f(X)$$ The differential of the trace of $F$ is then $$\eqalign{ d\,{\rm Tr}(F) &= f'(X)^T:dX \cr &= \big(I+\log(X)\big)^T:dX \cr }$$ Now assume that $X$ is described in terms of another matrix $L$, i.e. $$\eqalign{ X &= L^TL \cr dX &= L^T\,dL + dL^T\,L \cr }$$ Substituting this into the previous result yields $$\eqalign{ d\,{\rm Tr}(F) &= \big(I+\log(L^TL)\big)^T:(L^T\,dL + dL^T\,L) \cr &= 2\,\big(I+\log(L^TL)\big):L^T\,dL \cr &= 2\,L\big(I+\log(L^TL)\big):dL \cr \frac{\partial{\,\rm Tr}(F)}{\partial L} &= 2\,\big(L+L\log(L^TL)\big) \cr\cr }$$ NB: In some steps above, a colon is used as a product notation for the trace, i.e. $$A:B = {\rm Tr}(A^TB)$$