How is the rank of a matrix affected by centering the columns of a matrix? For some $n$ by $p$ matrix $X$, I'm trying to figure out how the rank of $X$ is affected if each column in $X$ is centered by the mean of that column (call the centered design matrix $Z$).
If $p < n$ and $X$ is full column rank, $Z$ is full column rank if multicollinearity is not present.
If $p = n$ and $X$ is full rank, $Z$ has rank $n-1$ due to the constraint from centering the variables, regardless of whether multicollinearity is present or not.
If $p > n$ and $X$ is full row rank, $Z$ has rank $n-1$ due to the constraint imposed from centering the variables
Which means rank of $Z \leq$ rank of $X$. I'm wondering if these observations are correct, and if so, if there's a technical way to show them, especially a).
 A: Your centered matrix is given by $Z= PX$ where $P:=I-\frac{1}{n}\mathbf{11}^T$.
Your 1st statement holds iff the ones vector is not in the column space of $X$. I.e. if $X\mathbf y = \mathbf 1$ then $PX\mathbf y = \mathbf 0$ and the kernel has dimension (at least) 1. Otherwise for any $X\mathbf y\neq \mathbf 1$ you have $P(X\mathbf y) = \mathbf 0$ iff $(X\mathbf y) = \mathbf 0$ which occurs iff $\mathbf y = \mathbf 0$ since $X$ has full column rank, so the kernel dimension is at most 1 as well.  Again the key issue is whether $\mathbf 1$ is in the column space of $X$.
As for your 2nd and 3rd points
having full row rank and at least as many rows as columns means the columns of $X$ span your space ($X$ is surjective) so the ones vector is in the column space of $X$ in both cases.  By the above argument $P$ acting on $X$ increments the kernel by 1 when we select $\mathbf y$ such that $X\mathbf y =\mathbf 1$ so $PX\mathbf y = \mathbf 0$.
To tighten this up, consider that
since $X$ is surjective it has a right inverse $M$ such that $XM = I_n$, then
$\text{rank}\big(PX\big) = \text{rank}\big(P(XM)X\big)\leq \text{rank}\big(P(XM)\big) =  \text{rank}\big(P\big) \leq \text{rank}\big(PX\big)$
so $\text{rank}\big(PX\big) =\text{rank}\big(P\big)=n-1$
