# How to get derivative of $\frac{d}{dX}(X^TX)$ and $\frac{d}{dX}(XX^T)$

How to get derivative of $\frac{d}{dX}(X^TX)$ and $\frac{d}{dX}(XX^T)$ ?

Where $X \in R^{n \times n}$. what is the difference between those two forms?

For matrix-valued functions I like to express derivatives in terms of the differential. If $\delta X$ is a variation of $X$, then $$d\left[X^TX\right](\delta X) = \delta X^TX + X^T\delta X,$$ where by $d[f(x)](\delta x)$ I mean the differential of $f$ applied to the $\delta x$ direction, i.e. the infinitesimal change in $f$ due to the infinitesimal change $\delta x$ in $x$; and $$d\left[XX^T\right](\delta X) = \delta XX^T + X\delta X^T.$$

In coordinates we have $$\frac{\partial (X^TX)}{\partial X_{kl}} = d[X^TX](e_ke_l^T) = e_le_k^TX + X^Te_ke_l^T = e_lX_k + (X_k)^T e_l^T$$ where $X_k$ is the $k$th row of $X$; and similarly $$\frac{\partial (XX^T)}{\partial X_{kl}} = d[XX^T](e_ke_l^T) = e_k e_l^T X^T + Xe_le_k^T = e_k (X^T)_l + \left[(X^T)_l\right]^Te_k^T,$$ and in particular these are not the same.

Hint: $$(X,Y)\longmapsto X^TY,\qquad (X,Y)\longmapsto XY^T$$ are bilinear and your functions are a composition $$X\longmapsto(X,X)\longmapsto X^TX,\qquad X\longmapsto(X,X)\longmapsto XX^T.$$ Now, apply the chain rule.

The short answer is: they're the same! To see why: the derivative of a matrix product has a very similar form to a derivative of a normal product:

${d \over dX}[f(X)g(X)] = f(X){d \over dX}[g(X)] + g(X){d \over dX}[f(x)]$

${d \over dX}[X^TX] = X^T{d \over dX}[X] + X{d \over dX}[X^T]$
${d \over dX}[XX^T] = X{d \over dX}[X^T] + X^T{d \over dX}[X]$
The way to calculate ${d \over dY}[X]$ when $X$ are $Y$ are same dimensions (for example, 2 by 2):
${d \over dY}[X] = \begin{bmatrix}{\partial X\over \partial Y_{11}} & {\partial X\over \partial Y_{12} } \\{\partial X\over \partial Y_{21}} & {\partial X\over \partial Y_{22}}\end{bmatrix}$
Where the $Y_{ij}$ are the scalar components of $Y$.