Proving that $H_1-H_0$ is idempotent I want to prove that for symmetric, idempotent matrices $H_1$ and $H_0$ (these are "hat matrices" of a linear regression model), $(H_1-H_0)=(H_1-H_0)^2$, in order to show a property of a distribution. So far, I've only found that the square equals $H_1-2H_0H_1+H_0$, where I have used the symmetry to change the order of matrix multiplication. But, would this not mean that I need to have $H_0-2H_0H_1=-H_0\iff H_0=H_0H_1\iff H_1=I$, which means that the matrix is, in fact, not idempotent, since $H_1$ is not necessarily equal to $I$? Thank you for your time.
 A: You need an additional hypothesis: The column space of $H_0$ is a subset of the column space of $H_1.$
If $H_0$ and $H_1$ are $n\times n$ symmetric idempotent matrices and the column space of $H_0$ is a subset of the column space of $H_1,$ then $H_0 H_1 = H_1 H_0 = H_0.$
If $x$ is in the column space of a symmetric idempotent real matrix $H,$ then $Hx=x,$ and if $x$ is orthogonal to the column space, then $Hx=0.$
If $x$ is any of the columns of $H_0$ and the aforementioned additional hypothesis holds, then $H_1 x = x.$ The columns of $H_1H_0$ are therefore just the columns of $H_0,$ so $H_1H_0= H_0.$ And since these matrices are symmetric, we also have $H_0 H_1=H_0.$
If $H_0$ had a right inverse matrix $A,$ then we could write:
$$
\require{cancel}
\xcancel{
\begin{align}
H_1 H_0 & = H_0. \\[6pt]
(H_1 H_0) A & = H_0 A = I. \\[6pt]
H_1 (H_0A) & = I. \\[6pt]
H_1 I & = I. \\[6pt]
H_1 & = I.
\end{align}}
$$
But no matrix with the same number of columns as rows has a one-sided inverse unless it has a two-sided inverse, and these don't.
A: Please, state the assumptions as it is done in (pure) linear algebra.
First of all, if two square matrices of the same dimension,$\,A\,\,B,\,$ are different then $\,A-B\,$ or $\,B-A\,$ is not idempotent.
For the time being, I still claim that the stated theorem is FALSE; that it is possible to have the two differences between two idempotent matrices $\,A\,B\,$ taken in either order, $\,A-B\,$ and
$\,B-A,\,$  neither has to be idempotent even in the case of 2x2 matrices which don't have $0'$s on the diagonal:
Let
$$ A\,\, =\,\, \left[\begin{array}{cc}
       \frac23 &  \frac23\\  \frac13 &  \frac13
   \end{array}\right] $$
and $\,B\,$ be the transpose of $\,A$:
$$ B\,\, =\,\, \left[\begin{array}{cc}
       \frac23 &  \frac13\\  \frac23 &  \frac13
   \end{array}\right] $$
Then both these matrices are idempotent, and
$$ A-B\,\, =\,\, \left[\begin{array}{cc}
       0 &  \frac13\\  -\frac13 &  0
   \end{array}\right] $$
while
$$ (A-B)^2\,\, =\,\, \left[\begin{array}{cc}
       -\frac19 &  0\\  0 & -\frac19
   \end{array}\right] $$
which means, indeed, that neither $\,A-B\,$ nor $\,B-A\,$ is
idempotent.   Great!

REMARK   There is an entire class of idempotent 2x2 matrices $\,A\,B\,$ such that $\,A-B\,$ or $\,B-A\,$ is idempotent hence I made sure to avoid this confusing situation.

