5
$\begingroup$

Given $X \in \mathbb{R}^{n \times r}$, what is $$\dfrac{dXX^T}{dX}?$$ I'm aware it is a order 4 tensor.

$\endgroup$
2
  • 2
    $\begingroup$ I don't know what the standard definitions or notations are in this context, but if we were to think of $dX$ as an infinitely small change in $X$ and $dXX^T$ as the corresponding change in $XX^T$, then I would guess that the value of the derivative at any point would be a linear transformation from the space of $n\times r$ matrices to the same of $n\times n$ matrices. If $A\in\mathbb R^{n\times n}$ and $B\in\mathbb R^{r\times n}$, then $A\,dX\,B\in\mathbb R^{n\times n}$ and every linear transformation from the first space to the second would be a sum of finitely many things of that form. $\endgroup$ Jul 14, 2014 at 22:52
  • $\begingroup$ Typo: "same of $n\times n$ matrices" ----> "space of $n\times n$ matrices". ${}\qquad{}$ $\endgroup$ Jul 15, 2014 at 3:02

2 Answers 2

5
$\begingroup$

$(X+H)(X+H)^{T}=XX^{T}+\underbrace{XH^{T}+HX^{T}}_{\text{first order terms}}+HH^{T}$

$\endgroup$
2
  • $\begingroup$ To add, in vectorized form this becomes $(I \otimes X)P + X \otimes I$, where $P$ is the vectorization transpose operator defined by $P \text{vec}(A) := \text{vec}(A^T)$, and $\otimes$ is the Kronecker product (matricized version of the tensor product). $\endgroup$
    – Nick Alger
    Jul 15, 2014 at 1:54
  • $\begingroup$ So then you'd have $dXX^T = X\ dX^T + dX\ X^T$. The actual value of the derivative would be the linear transformation from $\mathbb R^{n\times r}$ to $\mathbb R^{n\times n}$ that maps $A\in\mathbb R^{n\times r}$ to $XA^T+AX^T\in\mathbb R^{n\times n}$. ${}\qquad{}$ $\endgroup$ Jul 15, 2014 at 3:00
1
$\begingroup$

A pair of 4th order tensors arise naturally from matrix-matrix derivatives $$\eqalign{ \omega &= \frac {\partial{{\mathbf{X^T}}}} {\partial{{\mathbf{X}}}},\,\,\,\omega_{ijkl} = \delta_{il} \delta_{jk} \cr \varepsilon &= \frac {\partial{{\mathbf{X}}}} {\partial{{\mathbf{X}}}},\,\,\,\varepsilon_{ijkl} = \delta_{ik} \delta_{jl} \cr } $$ Using these, you can write the derivative as
$$\eqalign{ {\mathbf{\frac {\partial(X\cdot X^T)} {\partial X} }} &= {\mathbf{X\cdot\omega + \varepsilon\cdot X^T}} \cr } $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .