Idempotency of difference of two idempotent matrices

Define

$$\mathbf{H}=\mathbf{X}\left(\mathbf{X}^{\prime}\mathbf{X}\right)^{-1}\mathbf{X}^{\prime}$$

where $\mathbf{X}$ is of order $n \times k$

and

$$\overline{\mathbf{J}}=\frac{1}{n}\mathbf{J}=\frac{1}{n}\mathbf{1}\mathbf{1}^{\prime}$$

where $\mathbf{1}$ is a unit vector of order $n \times 1$.

Now $$\mathbf{H}\mathbf{H}=\mathbf{H}$$

and

$$\overline{\mathbf{J}}\overline{\mathbf{J}}=\overline{\mathbf{J}}$$

Thus both $\mathbf{H}$ and $\overline{\mathbf{J}}$ are idempotent matrices.

My question is whether $\mathbf{H}-\overline{\mathbf{J}}$ would be idempotent. If so then

$$\left(\mathbf{H}-\mathbf{\overline{\mathbf{J}}}\right)\mathbf{\left(\mathbf{H}-\mathbf{\overline{\mathbf{J}}}\right)}=\mathbf{H}-\mathbf{H\overline{\mathbf{J}}-\overline{\mathbf{J}}H}+\overline{\mathbf{J}}=\mathbf{H}-\mathbf{\overline{\mathbf{J}}-\overline{\mathbf{J}}}+\overline{\mathbf{J}}=\mathbf{H}-\overline{\mathbf{J}}$$

But I'm not able to show that

$$\mathbf{H\overline{\mathbf{J}}=\overline{\mathbf{J}}H}=\overline{\mathbf{J}}$$

I'd highly appreciate if you guide me to figure this. Thanks for your time and help.

-
If $\overline{\mathbf{J}}$ and $\mathbf{H}$ commute, you may diagonalise them and then I think the difference is again idempotent. –  draks ... Jan 5 '12 at 8:36
Andreas Spörl: It is not enough for them to commute. E.g., the difference is not idempotent when $\mathbf X$ is $n$-by-$1$ and orthogonal to $\mathbf 1$, even though $\mathbf{H}$ and $\overline{\mathbf{J}}$ commute in that case. –  Jonas Meyer Jan 5 '12 at 8:54

I'm assuming your scalars are real. I'm not sure what would happen over an arbitrary field.

Here $H$ and $J$ are $n \times n$, but presumably $X$ is $n \times m$ where $n > m$. Try an example with almost any $3 \times 2$ matrix $X$.

In general, $H$ is the orthogonal projection on the column space of $X$, while $\overline{J}$ is the orthogonal projection on the span of $\bf 1$. If $\bf 1$ is in the column space of $X$, then $H - \overline{J}$ is indeed an idempotent, the orthogonal projection on the orthogonal complement of $\bf 1$ in the column space of $X$. If not, then ${\bf 1}' (H - \overline{J}) {\bf 1} < 0$, which is impossible for an orthogonal projection (and any real symmetric idempotent is an orthogonal projection).

EDIT: over an arbitrary field whose characteristic does not divide $2n$, it's still true. Of course for $\overline{J}$ to exist, the characteristic can't divide $n$. A more general statement is: if $H$ and $K$ are symmetric idempotent matrices over a field not of characteristic $2$, then $H - K$ is an idempotent if and only if the column space of $K$ is contained in the column space of $H$.

If $H$ and $K$ are idempotents, $H - K$ is an idempotent if and only if $HK + KH = 2K$. Multiplying on the left by $K$, we get $KHK + KH = 2K$, so $HK = KHK$, and similarly multiplying on the right by $K$ we get $KH = KHK$. Thus $HK + KH = 2KHK = 2K$, so $HK = KH = K$ (here's where the characteristic can't be $2$). Thus the column space of $K$ is contained in the column space of $H$.

Conversely, if the column space of $K$ is contained in the column space of $H$, then $H K = K$ (i.e. for every vector $v$, $Kv = Hw$ for some $w$, and then $H K v = H H w = H w = K v$). If $H' = H$ and $K' = K$, this implies $K H = K' H' = (H K)' = K' = K$, and so $H - K$ is an idempotent.

-
(+1) Thanks @Robert for your elaborate answer. Much appreciated. –  MYaseen208 Jan 6 '12 at 2:36

${Added}$

I hastily assumed $\mathbf{X}$ to be a square invertible matrix before the question was edited. So this answer works only for that case.

Given $\mathbf{H}=\mathbf{X}(\mathbf{X'}\mathbf{X})^{-1}\mathbf{X'}$.

Thus, $\mathbf{H}=\mathbf{X}\mathbf{X^{-1}}\mathbf{(X')^{-1}}\mathbf{X'}=\mathbf{I}$ and $\mathbf{I}$ commutes with any matrix, thus it also commutes with $\bar{{\mathbf{J}}}$.

Hence, $\mathbf{H}\bar{{\mathbf{J}}}=\bar{{\mathbf{J}}}\mathbf{H}=\bar{{\mathbf{J}}}$ and we are done.

This works fine iff $\mathbf{X}$ is a square matrix- thanks to Jonas Meyer who pointed out the limitations of my earlier claim.

-
See my edited question. –  MYaseen208 Jan 5 '12 at 8:55
@JonasMeyer:- In the case when $\mathbf{X}$ is a column vector, $\mathbf{H}$ will be a scalar anyway. I will include this in the answer, thanks for pointing it out. –  NikBels Jan 5 '12 at 9:02
@Nikhil: Your answer works only when $k=n$, and $\mathbf X$ is an invertible matrix. –  Jonas Meyer Jan 5 '12 at 9:03
@Nikhil: No, if $\mathbf X$ is a column vector, then $\mathbf H$ is the rank one projection onto the span of $\{\mathbf X\}$. In general $\mathbf H$ is the projection onto the column space of $\mathbf X$. –  Jonas Meyer Jan 5 '12 at 9:04
@JonasMeyer Grr got the hitch now. Thanks for the clarification. –  NikBels Jan 5 '12 at 9:10

Added: Before the question edit, I hastily assumed that $\mathbf X$ is a nonzero column vector. So the answer at bottom only concerns the case where $k=1$. In the general case, it holds if $\mathbf 1$ is in the column space of $\mathbf X$ (see Robert Israel's answer).

In general, if $P$ and $Q$ are self-adjoint projections on an inner product space $V$, then $P-Q$ is a projection if and only if $QV\subseteq PV$ if and only if $PQ=QP=Q$.

Consider the special case $k=1$. Then it is true if and only if $\mathbf{X}$ is a nonzero scalar multiple of $\mathbf{1}$ (which I presume is the column vector with all $1$s). If $\mathbf X=\lambda\mathbf 1$ for some $\lambda\neq 0$, then $\mathbf H=\overline{\mathbf J}$, so $\mathbf H-\overline{\mathbf J}=0$.

Suppose that $\mathbf X$ is not a scalar multiple of $\mathbf 1$. Then $\|\langle \mathbf 1,\mathbf X\rangle\|<\|\mathbf 1\|\|\mathbf X\|$. Therefore $\|\mathbf{H}\overline{\mathbf{J}}\mathbf1\|=\|\mathbf H\mathbf 1\|=\left\|\frac{\langle \mathbf 1,\mathbf X\rangle}{\|\mathbf X\|^2}\mathbf X\right\|<\|\mathbf 1\|$. This implies that $\mathbf H\overline{\mathbf J}+\overline{\mathbf J}\mathbf H\neq 2\overline{\mathbf J}$.

-
Nikhil's answer makes your look kinda stupid now. Not that it's wrong, just too elaborate for something simple as that. –  Patrick Da Silva Jan 5 '12 at 8:56
@Patrick: Our answers are saying very different things. Both were based on our mistakes of answering the question before getting clarification on what $X$ is. I assumed it was a nonzero column vector, while Nikhil's answer has something different entirely. –  Jonas Meyer Jan 5 '12 at 8:58
Hm. I didn't notice that either. Sorry for my comment. –  Patrick Da Silva Jan 5 '12 at 9:12