# Proof that trace of 'hat' matrix in linear regression is rank of X

I understand that the trace of the projection matrix (also known as the "hat" matrix) X*Inv(X'X)*X' in linear regression is equal to the rank of X. How can we prove that from first principles, i.e. without simply asserting that the trace of a projection matrix always equals its rank?

I am aware of the post Proving: "The trace of an idempotent matrix equals the rank of the matrix", but need an integrated proof.

• Does the identity $\text{tr}(AB) = \text{tr}(BA)$ count as being from first principles? – JimmyK4542 Dec 19 '15 at 21:58
• I suppose so. One can write that out with indices and see that it holds true. – ClarPaul Dec 19 '15 at 22:28
• Oh, so, one writes: tr(X*in(X'X)*X')=tr(X'*X*inv(X'X))=tr(I)=dim(X'X) !! Great, thanks! – ClarPaul Dec 19 '15 at 22:38

If $X$ is $n \times m$ with $m \le n$ and has full rank, then $rank (X) = \min(n,m) = m$, and we know $(X^T X)^{-1}$ exists.
$$tr(H) := tr (X (X^T X)^{-1} X^T) = tr (X^T X (X^T X)^{-1} ) = tr[I_m] = m$$
Let $X$ be an $n \times r$ matrix with full rank, i.e. $\text{rank}X = r$ (this is necessary for $(X^TX)^{-1}$ to exist).
Then, using the identity $\text{tr}(AB) = \text{tr}(BA)$, we have
$\text{tr}[X(X^TX)^{-1}X^T] = \text{tr}[X^TX(X^TX)^{-1}] = \text{tr}[I_{r \times r}] = r = \text{rank}X$, as desired.