Jacobian of the row de-meaning of a matrix X, with respect to matrix X

Question

Let $\mathbf{X}$ $\in \mathbb{R}^{M \times N}$ be a long matrix, with $M<N$.

Let $\mathbf{Y}$ $\in \mathbb{R}^{M \times N}$ represent $\mathbf{X}$ after each row of $\mathbf{X}$ has been "de-meaned", such that $\mathbf{Y}$ $\mathbf{1}_N^\top$ $=$ $\mathbf{0}_M$. Here, $\mathbf{1}_N$ $\in \mathbb{R}^{N}$ is a vector of $1$s, and $\mathbf{0}_M$ $\in \mathbb{R}^{M}$ is a vector of $0$s.

I want to determine the Jacobian of $\mathbf{Y}$ with respect to $\mathbf{X}$, i.e. the quantity $\frac{\partial \text{vec}(\mathbf{Y})}{\partial \text{vec}(\mathbf{X})}$ $\in \mathbb{R}^{MN \times MN}$ .

Since it is often convenient to use the Kronecker product in the Jacobian, it may be most convenient to explicitly write $\mathbf{Y}$ via the expression:

$\mathbf{Y}$ $=$ $\mathbf{X}$ $-$ $\bigg[ (\frac{1}{N} \ \mathbf{X} \ \mathbf{1}_N^\top) \ \bigotimes \ \mathbf{1}_N^\top \bigg]$

where $\frac{1}{N} \ \mathbf{X} \ \mathbf{1}_N^\top$ $\in$ $\mathbb{R}^{M}$ is the vector containing the means of each row in $\mathbf{X}$.

My concern is how to process $\bigg[ (\frac{1}{N} \ \mathbf{X} \ \mathbf{1}_N^\top) \ \bigotimes \ \mathbf{1}_N^\top \bigg]$ in the calculation of the Jacobian.

Answer 1

$ \def\d{\delta} \def\l{\ell} \def\k{\otimes} \def\o{{\tt1}} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\vc#1{\op{vec}\LR{#1}} \def\q{\quad} \def\qq{\qquad} \def\qiq{\q\implies\q} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} $Let $C$ denote the Centering Matrix and write the $Y$ matrix as $$\eqalign{ Y &= XC \qiq dY = dX\;C \\ }$$ Now there are several options to calculate the gradient, one of which is vectorization $$\eqalign{ \vc{dY} &= \LR{C\k I}\vc{dX} \qiq \grad{\vc Y}{\vc X} = C\k I \\ }$$ Another approach is differentiation with respect to the components of $X$ $$\eqalign{ \grad Y{X_{k\l}} = E_{k\l}\,C \\ }$$ where $E_{kl}$ is a matrix whose elements are all zero except for a $\o$ in the $(k,l)$ position.

Going further, the full component-wise gradient can be written as $$\eqalign{ \grad{Y_{ij}}{X_{k\l}} = \d_{ik}\,C_{j\l} \\ }$$