Show that if $X = [X_1 $ $X_2]$ and $X_1'X_2=0$, then $P = P_1+P_2$ (where $P = X(X'X)^{-1}X'$) I am trying to express $P$ in terms of $X_1$ and $X_2$. Is it possible to do so?
 A: Suppose
$$
X_1\in \mathbb R^{n\times k},\quad X_2\in\mathbb R^{n\times\ell}, \text{ so }\begin{bmatrix} X_1,X_2 \end{bmatrix}\in\mathbb R^{n\times(k+\ell)}.
$$
Then
$$
X'X = \begin{bmatrix} X_1' \\  X_2' \end{bmatrix} \begin{bmatrix} X_1, & X_2 \end{bmatrix} = \begin{bmatrix} X_1'X_1, & X_1'X_2 \\  X_2'X_1, & X_2'X_2 \end{bmatrix} \in \mathbb R^{(k+\ell)\times(k+\ell)}.
$$
But this is equal to
$$
\begin{bmatrix} X_1'X_1, & 0 \\  0, & X_2'X_2 \end{bmatrix} \in \mathbb R^{(k+\ell)\times(k+\ell)},
$$
and so
$$
\begin{bmatrix} X_1'X_1, & 0 \\  0, & X_2'X_2 \end{bmatrix}^{-1} = \begin{bmatrix} (X_1'X_1)^{-1}, & 0 \\  0, & (X_2'X_2)^{-1} \end{bmatrix}.
$$
Then
$$
\begin{bmatrix} X_1 &  X_2 \end{bmatrix} \begin{bmatrix} (X_1'X_1)^{-1}, & 0 \\  0, & (X_2'X_2)^{-1} \end{bmatrix} \begin{bmatrix} X_1'  \\ X_2' \end{bmatrix} = X_1(X_1 X_1)^{-1} X_1' + X_2(X_2 X_2)^{-1} X_2'
$$
$$
= P_1 + P_2.
$$
You just multiply matrices with matrix entries the way you multiply matrices with scalar entries, except that in every multiplication, it matters which factor is on the left and which is on the right.
