# What is $\frac{dXX^T}{dX}$?

Given $X \in \mathbb{R}^{n \times r}$, what is $$\dfrac{dXX^T}{dX}?$$ I'm aware it is a order 4 tensor.

• I don't know what the standard definitions or notations are in this context, but if we were to think of $dX$ as an infinitely small change in $X$ and $dXX^T$ as the corresponding change in $XX^T$, then I would guess that the value of the derivative at any point would be a linear transformation from the space of $n\times r$ matrices to the same of $n\times n$ matrices. If $A\in\mathbb R^{n\times n}$ and $B\in\mathbb R^{r\times n}$, then $A\,dX\,B\in\mathbb R^{n\times n}$ and every linear transformation from the first space to the second would be a sum of finitely many things of that form. Jul 14, 2014 at 22:52
• Typo: "same of $n\times n$ matrices" ----> "space of $n\times n$ matrices". ${}\qquad{}$ Jul 15, 2014 at 3:02

$(X+H)(X+H)^{T}=XX^{T}+\underbrace{XH^{T}+HX^{T}}_{\text{first order terms}}+HH^{T}$
• To add, in vectorized form this becomes $(I \otimes X)P + X \otimes I$, where $P$ is the vectorization transpose operator defined by $P \text{vec}(A) := \text{vec}(A^T)$, and $\otimes$ is the Kronecker product (matricized version of the tensor product). Jul 15, 2014 at 1:54
• So then you'd have $dXX^T = X\ dX^T + dX\ X^T$. The actual value of the derivative would be the linear transformation from $\mathbb R^{n\times r}$ to $\mathbb R^{n\times n}$ that maps $A\in\mathbb R^{n\times r}$ to $XA^T+AX^T\in\mathbb R^{n\times n}$. ${}\qquad{}$ Jul 15, 2014 at 3:00
A pair of 4th order tensors arise naturally from matrix-matrix derivatives \eqalign{ \omega &= \frac {\partial{{\mathbf{X^T}}}} {\partial{{\mathbf{X}}}},\,\,\,\omega_{ijkl} = \delta_{il} \delta_{jk} \cr \varepsilon &= \frac {\partial{{\mathbf{X}}}} {\partial{{\mathbf{X}}}},\,\,\,\varepsilon_{ijkl} = \delta_{ik} \delta_{jl} \cr } Using these, you can write the derivative as
\eqalign{ {\mathbf{\frac {\partial(X\cdot X^T)} {\partial X} }} &= {\mathbf{X\cdot\omega + \varepsilon\cdot X^T}} \cr }