6
$\begingroup$

For some $n$ by $p$ matrix $X$, I'm trying to figure out how the rank of $X$ is affected if each column in $X$ is centered by the mean of that column (call the centered design matrix $Z$).

If $p < n$ and $X$ is full column rank, $Z$ is full column rank if multicollinearity is not present.

If $p = n$ and $X$ is full rank, $Z$ has rank $n-1$ due to the constraint from centering the variables, regardless of whether multicollinearity is present or not.

If $p > n$ and $X$ is full row rank, $Z$ has rank $n-1$ due to the constraint imposed from centering the variables

Which means rank of $Z \leq$ rank of $X$. I'm wondering if these observations are correct, and if so, if there's a technical way to show them, especially a).

$\endgroup$
0

1 Answer 1

7
$\begingroup$

Your centered matrix is given by $Z= PX$ where $P:=I-\frac{1}{n}\mathbf{11}^T$.

Your 1st statement holds iff the ones vector is not in the column space of $X$. I.e. if $X\mathbf y = \mathbf 1$ then $PX\mathbf y = \mathbf 0$ and the kernel has dimension (at least) 1. Otherwise for any $X\mathbf y\neq \mathbf 1$ you have $P(X\mathbf y) = \mathbf 0$ iff $(X\mathbf y) = \mathbf 0$ which occurs iff $\mathbf y = \mathbf 0$ since $X$ has full column rank, so the kernel dimension is at most 1 as well. Again the key issue is whether $\mathbf 1$ is in the column space of $X$.

As for your 2nd and 3rd points
having full row rank and at least as many rows as columns means the columns of $X$ span your space ($X$ is surjective) so the ones vector is in the column space of $X$ in both cases. By the above argument $P$ acting on $X$ increments the kernel by 1 when we select $\mathbf y$ such that $X\mathbf y =\mathbf 1$ so $PX\mathbf y = \mathbf 0$.

To tighten this up, consider that
since $X$ is surjective it has a right inverse $M$ such that $XM = I_n$, then
$\text{rank}\big(PX\big) = \text{rank}\big(P(XM)X\big)\leq \text{rank}\big(P(XM)\big) = \text{rank}\big(P\big) \leq \text{rank}\big(PX\big)$
so $\text{rank}\big(PX\big) =\text{rank}\big(P\big)=n-1$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .