Proof that trace of 'hat' matrix in linear regression is rank of X

Question

I understand that the trace of the projection matrix (also known as the "hat" matrix) X*Inv(X'X)*X' in linear regression is equal to the rank of X. How can we prove that from first principles, i.e. without simply asserting that the trace of a projection matrix always equals its rank?

I am aware of the post Proving: "The trace of an idempotent matrix equals the rank of the matrix", but need an integrated proof.

Answer 1

If $X$ is $n \times m$ with $m \le n$ and has full rank, then $rank (X) = \min(n,m) = m$, and we know $(X^T X)^{-1}$ exists.

By commutativity of the trace operator, we have

$$tr(H) := tr (X (X^T X)^{-1} X^T) = tr (X^T X (X^T X)^{-1} ) = tr[I_m] = m$$

Answer 2

Let $X$ be an $n \times r$ matrix with full rank, i.e. $\text{rank}X = r$ (this is necessary for $(X^TX)^{-1}$ to exist).

Then, using the identity $\text{tr}(AB) = \text{tr}(BA)$, we have

$\text{tr}[X(X^TX)^{-1}X^T] = \text{tr}[X^TX(X^TX)^{-1}] = \text{tr}[I_{r \times r}] = r = \text{rank}X$, as desired.