# partial derivative of transpose matrix-matrix multiplication

I came across some problems that are related to partial derivative but I haven't learnt this yet. And I looked up many online resources but couldn't find answers to my doubts. Really hope someone can help me.
Here is my problem. $$y=A^TB$$, where A, B are two matrices. Now I want to know what $$\frac{\partial y}{\partial A}$$,$$\frac{\partial y}{\partial B}$$ are.

• No, your problem isn't this. Your problem is that partial derivatives with respect to a single (real) variable are well defined (and you may or may be not aware of that definition), but there is no commonly accepted notion/notation for a vector/matrix variable. Please, "research" does not mean "ask at MSE"! Would you be so kind to do a minimum of own effort? Thank you ever so much! – user436658 Jan 13 at 14:11

$$\def\p#1#2{\frac{\partial #1}{\partial #2}}$$Let $$\,(\alpha,\beta)\,$$ be fourth-order tensors with components \eqalign{ \alpha_{ijk\ell} &= \delta_{ik}\,\delta_{j\ell} \\ \beta_{ijk\ell} &= \delta_{i\ell}\,\delta_{jk} \\ } and properties with respect to the matrices $$(F,G,H)$$ \eqalign{ \alpha:H &= H:\alpha = H \\ \beta:F &= F:\beta = F^T \\ HFG &= H\alpha G^T:F \\ } where a colon denotes a double-contraction product, i.e. \eqalign{ \left(\alpha:H\right)_{ij} &= \sum_k\sum_\ell\alpha_{ijk\ell}\,H_{k\ell} \\ \left(F:\beta\right)_{k\ell} &= \sum_i\sum_jF_{ij}\,\beta_{ijk\ell} \\ } and juxtaposition implies a single-contraction product \eqalign{ \left(H\alpha\right)_{mjk\ell} &= \sum_i H_{mi}\,\alpha_{ijk\ell} \\ \left(\alpha G^T\right)_{ijkm} &= \sum_\ell\alpha_{ijk\ell}\,G^T_{\ell m} \\ }
With these tensors, the posted question can be answered as follows \eqalign{ Y &= A^TB \\&= A^T\alpha:B \quad&\implies\quad\p{Y}{B} &= A^T\alpha \\ Y &= \alpha B^T:A^T \\ &= \alpha B^T:\beta:A \quad&\implies\quad\p{Y}{A} &= \alpha B^T:\beta \\ } So the gradients in question are seen to be fourth-order tensors.
An approach which avoids higher-order tensors, is to transform the relationship into a vector equation using Kronecker products. \eqalign{ {\rm vec}(Y) &= (I\otimes A^T)\;{\rm vec}(B) \quad&\implies\quad \p{{\,\rm vec}\,Y}{{\,\rm vec}\,B} = (I\otimes A^T) \\ &= (B^T\otimes I)K\;{\rm vec}(A) \quad&\implies\quad \p{{\,\rm vec}\,Y}{{\,\rm vec}\,A} = (B^T\otimes I)K \\ } where $$K$$ is the commutation matrix associated with vectorization.
Another approach is to use component-wise derivatives \eqalign{ \p{Y}{A_{ij}} &= E_{ij}^TB \qquad\quad \p{Y}{B_{ij}} &= A^TE_{ij} \\ } where $$E_{ij}$$ is a matrix with all components equal to zero, except the $$(i,j)$$ component which equals one. And any matrix with independent components satisfies the identity \eqalign{ \p{G}{G_{k\ell}} &= E_{k\ell} \qquad\iff\qquad \p{G^T}{G_{k\ell}} &= E_{k\ell}^T \\ } Finally, to bring things full circle \eqalign{ \p{G_{ij}}{G_{k\ell}} &= \alpha_{ijk\ell} \qquad\iff\qquad \p{G_{ij}^T}{G_{k\ell}} &= \beta_{ijk\ell} \\ }