# Hat Matrix Identities in Regression

I need to show that $\bar h= \sum{h_{ii}/n} = \operatorname{Tr}[H]/n = (p+1)/n$

Using the fact that $\operatorname{Tr}[AB]=\operatorname{Tr}[BA]$ and $H=X(X^TX)^{-1}X^T$.

But I have no idea how to calculate $\bar h$, I'm betting the first equality works out because $H$ is a symmetric idempotent matrix. I also have no clue what $\operatorname{Tr}[H]$ means, I have never seen this notation before an cannot find it in my notes.

-

The trace of a matrix is the sum of its diagonal entries. The function $A\mapsto \text{Tr}(A)$ has the property that $\text{Tr}(AB)=\text{Tr}(BA)$ for any two matrices of compatible size.
So, if $X$ is of size $n\times(p+1)$, then $X^TX$ is a $(p+1)\times(p+1)$ matrix and $$\text{Tr}(H)=\text{Tr}(X(X^TX)^{-1}X^T)=\text{Tr}((X^TX)^{-1}X^TX)=\text{Tr}(I_{p+1})=p+1.$$
So $\sum_{H_{jj}}/n=\text{Tr}(H)/n=(p+1)/n$.
Sorry!!! Typically in class we define P as the highest subscript of $\beta$. So $\beta_0, \beta_1, ..., \beta_p$ – Carly Nov 24 '12 at 21:48
From what I recall, $X^T X$ has dimension $(p+1) \times (p+1)$, i.e. dimension of the vector $\beta$. So that $\operatorname{Tr}(H)$ should be equal to $(p+1)$, not $n$. – johnny Nov 24 '12 at 21:49
@MartinArgerami : It is conventional in dealing with certain design matrices in statistics to let $p$ be the number of columns other than one column whose every entry is $1$. Hence $X$ has $p+1$ columns, and typically a much larger number $n$ of rows than columns. So the "hat matrix" $H$ is an $n\times n$ matrix of rank $p+1$. If $Y\in\mathbb{R}^{n\times1}$ then $HY$ is the orthogonal projection of $Y$ onto the column space of $X$. – Michael Hardy Nov 24 '12 at 22:27
There is a grave error in this answer. If $X\in\mathbb R^{n\times(p+1)}$ then the identity matrix at the end is a $(p+1)\times(p+1)$ identity matrix. Its trace is $p+1$. – Michael Hardy Nov 24 '12 at 22:30
@Carly : Please: Don't write capital $P$ when you mean lower-case $p$. Mathematical notation is case-sensitive. – Michael Hardy Nov 24 '12 at 22:30