0
$\begingroup$

How to get derivative of $\frac{d}{dX}(X^TX) $ and $\frac{d}{dX}(XX^T) $ ?

Where $X \in R^{n \times n}$. what is the difference between those two forms?

$\endgroup$

3 Answers 3

1
$\begingroup$

For matrix-valued functions I like to express derivatives in terms of the differential. If $\delta X$ is a variation of $X$, then $$d\left[X^TX\right](\delta X) = \delta X^TX + X^T\delta X,$$ where by $d[f(x)](\delta x)$ I mean the differential of $f$ applied to the $\delta x$ direction, i.e. the infinitesimal change in $f$ due to the infinitesimal change $\delta x$ in $x$; and $$d\left[XX^T\right](\delta X) = \delta XX^T + X\delta X^T.$$

In coordinates we have $$\frac{\partial (X^TX)}{\partial X_{kl}} = d[X^TX](e_ke_l^T) = e_le_k^TX + X^Te_ke_l^T = e_lX_k + (X_k)^T e_l^T $$ where $X_k$ is the $k$th row of $X$; and similarly $$\frac{\partial (XX^T)}{\partial X_{kl}} = d[XX^T](e_ke_l^T) = e_k e_l^T X^T + Xe_le_k^T = e_k (X^T)_l + \left[(X^T)_l\right]^Te_k^T, $$ and in particular these are not the same.

$\endgroup$
0
$\begingroup$

Hint: $$ (X,Y)\longmapsto X^TY,\qquad (X,Y)\longmapsto XY^T $$ are bilinear and your functions are a composition $$ X\longmapsto(X,X)\longmapsto X^TX,\qquad X\longmapsto(X,X)\longmapsto XX^T. $$ Now, apply the chain rule.

$\endgroup$
0
$\begingroup$

The short answer is: they're the same! To see why: the derivative of a matrix product has a very similar form to a derivative of a normal product:

${d \over dX}[f(X)g(X)] = f(X){d \over dX}[g(X)] + g(X){d \over dX}[f(x)]$

So for your two functions:

${d \over dX}[X^TX] = X^T{d \over dX}[X] + X{d \over dX}[X^T]$

${d \over dX}[XX^T] = X{d \over dX}[X^T] + X^T{d \over dX}[X]$

As you can see, the only difference is the order in which you add them. But matrix addition is commutative.

The way to calculate ${d \over dY}[X]$ when $X$ are $Y$ are same dimensions (for example, 2 by 2):

${d \over dY}[X] = \begin{bmatrix}{\partial X\over \partial Y_{11}} & {\partial X\over \partial Y_{12} } \\{\partial X\over \partial Y_{21}} & {\partial X\over \partial Y_{22}}\end{bmatrix}$

Where the $Y_{ij}$ are the scalar components of $Y$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .