4
$\begingroup$

For some $n$ by $p$ matrix $X$, I'm trying to figure out how the rank of $X$ is affected if each column in $X$ is centered by the mean of that column (call the centered design matrix $Z$).

If $p < n$ and $X$ is full column rank, $Z$ is full column rank if multicollinearity is not present.

If $p = n$ and $X$ is full rank, $Z$ has rank $n-1$ due to the constraint from centering the variables, regardless of whether multicollinearity is present or not.

If $p > n$ and $X$ is full row rank, $Z$ has rank $n-1$ due to the constraint imposed from centering the variables

Which means rank of $Z \leq$ rank of $X$. I'm wondering if these observations are correct, and if so, if there's a technical way to show them, especially a).

$\endgroup$
0

1 Answer 1

5
$\begingroup$

Your centered matrix is given by $Z= PX$ where $P:=I-\frac{1}{n}\mathbf{11}^T$.

Your 1st statement holds iff the ones vector is not in the column space of $X$. I.e. if $X\mathbf y = \mathbf 1$ then $PX\mathbf y = \mathbf 0$ and the kernel has dimension (at least) 1. Otherwise for any $X\mathbf y\neq \mathbf 1$ you have $P(X\mathbf y) = \mathbf 0$ iff $(X\mathbf y) = \mathbf 0$ which occurs iff $\mathbf y = \mathbf 0$ since $X$ has full column rank, so the kernel dimension is at most 1 as well. Again the key issue is whether $\mathbf 1$ is in the column space of $X$.

As for your 2nd and 3rd points
having full row rank and at least as many rows as columns means the columns of $X$ span your space ($X$ is surjective) so the ones vector is in the column space of $X$ in both cases. By the above argument $P$ acting on $X$ increments the kernel by 1 when we select $\mathbf y$ such that $X\mathbf y =\mathbf 1$ so $PX\mathbf y = \mathbf 0$.

To tighten this up, consider that
since $X$ is surjective it has a right inverse $M$ such that $XM = I_n$, then
$\text{rank}\big(PX\big) = \text{rank}\big(P(XM)X\big)\leq \text{rank}\big(P(XM)\big) = \text{rank}\big(P\big) \leq \text{rank}\big(PX\big)$
so $\text{rank}\big(PX\big) =\text{rank}\big(P\big)=n-1$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.