Finding the Gradient Matrix for the given expression

Let $$\rho$$ be a matrix, and let $$\rho_A$$ be the partial trace of the matrix $$\rho$$. For simplicity, let us assume $$\rho$$ is a $$4 \times 4$$ matrix. The partial trace is defined as follows:

If $$\rho = \begin{bmatrix}a & b & c & d \\ e & f & g & h \\ i & j & k & l \\ m & n & o & p \end{bmatrix}$$

Then $$\rho_A = \begin{bmatrix} a+f & c+h \\i+n & k + p \end{bmatrix}$$

I would like to calculate the following: $$\nabla_\rho (Tr(\rho log \rho) - Tr(\rho_A log \rho_A))$$

which is the derivative of a continuous scalar quantity with respect to a matrix, so I should be able to calculate it right?

Note that each of the entries are complex.

I tried a few things, but I'm not sure whether chain rule works, because you end up getting gradients of matrices with respect to matrices, which is not defined?

Any help on how to go about such a thing would be very appreciated.

• You should find this this post to be useful – Omnomnomnom Aug 12 at 17:11
• If you write the components of your gradient in terms of partial derivatives, then the chain rule should work – stochastic Aug 12 at 17:49

Consider a scalar function, its derivative, and its differential. \eqalign{ \phi &= {\rm Tr}\big(\alpha \log(\alpha)\big) &\implies \frac{d\phi}{d\alpha} = \log(\alpha) + 1 \cr d\phi &= \big(\log(\alpha) + 1\big)\,d\alpha \cr } When the argument is a square matrix $$(A)$$, this becomes \eqalign{ \phi &= {\rm Tr}\big(A\log(A)\big) \cr d\phi &= \big(\log(A)+I\,\big)^T:dA \cr } where a colon denotes the trace product, i.e. $$\,A:B = {\rm Tr}(A^TB)$$

Now define the cartesian basis vectors and their matrix analogs. \eqalign{ &e_1 &= \pmatrix{1\\0},\quad &e_2 &= \pmatrix{0\\1} \cr &E_1 &= e_1\otimes I,\quad &E_2 &= e_2\otimes I \cr } where $$I\in{\mathbb R}^{2\times 2}$$ is the identity matrix.

The basis matrices can be used to extract $$2\times 2$$ blocks from the $$\rho$$ matrix while the basis vectors can be used to construct the components of $$\rho_A$$. \eqalign{ \rho_A &= e_1e_1^T\;{\rm Tr}\big(E_1^T\rho E_1\big) + e_1e_2^T\;{\rm Tr}\big(E_1^T\rho E_2\big) + e_2e_1^T\;{\rm Tr}\big(E_2^T\rho E_1\big) + e_2e_2^T\;{\rm Tr}\big(E_2^T\rho E_2\big) \cr \rho_A &= e_ie_j^T\big(E_iE_j^T:\rho\big) } Invoking the summation convention over $$(i,j)$$ produces a concise expression.

Combining the above results answers the question that was posed. \eqalign{ \psi &= {\rm Tr}\big(\rho\log(\rho)\big) - {\rm Tr}\big(\rho_A\log(\rho_A)\big) \cr d\psi &= \big(\log(\rho)^T+I\otimes I\big):d\rho - \big(\log(\rho_A)^T+I\big):d\rho_A \cr &= \big(\log(\rho)^T+I\otimes I\big):d\rho - \big(\log(\rho_A)^T+I\big):e_ie_j^T\big(E_iE_j^T:d\rho\big) \cr &= \big(\log(\rho)^T+I\otimes I\big):d\rho - \big(e_i^T\log(\rho_A)^Te_j+\delta_{ij}\big)\big(E_iE_j^T:d\rho\big) \cr \frac{\partial\psi}{\partial \rho} &= \log(\rho)^T + I\otimes I - e_i^T\log(\rho_A)^Te_j\,E_iE_j^T - E_jE_j^T \\ &= \log(\rho)^T - e_i^T\log(\rho_A)^Te_j\,E_iE_j^T \\ &= \log(\rho)^T - e_i^T\log(\rho_A)^Te_j\,\big(e_ie_j^T\otimes I\big) \\ &= \Big(\log(\rho) - \log(\rho_A)\otimes I\Big)^T \\ }

• Thank you very much! In my matrices however, all the entries are complex? Does that change anything? For instance, inner product when matrices are complex is generally written as $Tr(A^\dagger B)$. – Mahathi Vempati Aug 13 at 8:05
• Also, do you know a good source where I can learn these things? I looked at some matrix differentiation slides/ notes, but never came across using the inner product in this way. – Mahathi Vempati Aug 13 at 8:15
• Note that I'm not using what others would call the matrix inner product, I'm simply using the colon as a convenient product notation for the trace function. So everything that I've done above is consistent with complex matrices. – greg Aug 13 at 12:56
• One nice thing about the colon product is that, since no complex conjugation is involved, it's commutative, i.e. $$A:B = B:A$$ This is not the case for the inner product. – greg Aug 13 at 15:01
• Ah, that's nice. What I meant was how do we know that the derivative of a square matrix maps to a scalar, and that trace product should be used to multiply $dA$, for instance. These maps don't seem intuitive to me. Do you know where I can learn these things, like maybe a textbook reference? – Mahathi Vempati Aug 13 at 15:05