1
$\begingroup$

Notation: for a matrix $Z$ of full column rank, we define $P(Z)=Z(Z'Z)^{-1}Z'$ and $M(Z)=I-P(Z)$. ($I$ is the identity of matrix with as many rows as $Z$.)

Let's consider the linear regression model with fixed regressors: $$ y=\underbrace{X_1}_{T\times k_1}\beta_1+\underbrace{X_2}_{T\times k_2}\beta_2+u,\quad u\sim N(0,\sigma^2 I_T). $$ With $X=(X_1\;\; X_2)$ (dimension $n\times(k_1+k_2)$), the $F$ statistic for the null hypothesis $\beta_1=\gamma$ can be written in the following equivalent forms: (below $\hat{\beta}_1$ denotes the OLS estimator of $\beta_1$) \begin{align*} F&\equiv\frac{(\hat\beta_1-\gamma)'X_2'M(X_1)X_2(\hat\beta_1-\gamma)/k_1}{y'M(X)y/(T-k_1-k_2)}\\ &=\frac{u'(M(X_2)-M(X))u/k_1}{u'M(X)u/(T-k_1-k_2)}\\ &=\frac{u'P(M(X_2)X_1)u/k_1}{u'M(X)u/(T-k_1-k_2)}\cdot \end{align*} Given the equality of the expressions on the second and third lines above, I suspect $$ M(X_2)-M(X)\overset{?}{=}P(M(X_2)X_1)=M(X_2)X_1(X_1'M(X_2)X_1)^{-1}X_1'M(X_2).\tag{$*$} $$ I tried proving it with various algebraic manipulations but have been hitting dead ends so far. Thanks for your help.


Edit: Writing $M(X_2)=I-P(X_2)$, we can transform ($*$) as $$ M(X_2)-M(X)=P(M(X_2)X_1)\iff P(X)=P(X_2)+P(M(X_2)X_1)\tag{$**$}. $$ ($**$) has a geometric interpretation: projecting on the column space of $X_1$ and $X_2$ can be done in 2 steps: projecting first on $X_2$ and add that to the projection on the "marginal" contribution of $X_1$.

$\endgroup$
0
$\begingroup$

I think I've found a proof. Using the block inversion formula, we can write $$ P(X)=X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)+X_2(X_2'M(X_1)X_2)^{-1}X_2M(X_1). $$ Because $I=P(X_2)+M(X_2)$ and $X_2=P(X_2)X_2$, we can write \begin{align*} P(X)&=M(X_2)X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)+P(X_2)X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)\\ &\quad+P(X_2)X_2(X_2'M(X_1)X_2)^{-1}X_2M(X_1)\\ &=P(M(X_2)X_1)\\ &\quad+P(X_2)[X_1(X_1'M(X_2)X_1)^{-1}X_1M(X_2)+X_2(X_2'M(X_1)X_2)^{-1}X_2M(X_1)]\\ &=P(M(X_2)X_1)+P(X_2)P(X)\\ &=P(M(X_2)X_1)+P(X_2) \end{align*} where the last equality is because $$ P(X)P(X_2)=P(X)X_2(X_2'X_2)^{-1}X_2=X_2(X_2'X_2)^{-1}X_2=P(X_2). $$ I'll leave this answer here because it seems correct but it's very grungy because it involves the block inversion formula. I hope to learn a more elegant proof from the community.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.