On the projection matrix on sum of two subspaces Define $Z=(X:Y)$. We know that the projection matrix of $Z$ (say, $P_z$) can be expressed as the sum of the projection matrix of $X$ (say, $P_x$) and that of $(I-P_x)Y$. If the column spaces of $X$ and $Y$ are orthogonal, then $P_z=P_x+P_y$, where $P_y$ is the projection matrix of $Y$.
However, I am interested in the general case when the column spaces of $X$ and $Y$ are not necessarily orthogonal. In this context, is it possible to write $P_z \leq P_x+P_y$? Here, `$A\leq B$' implies that $(B-A)$ is a non-negative definite matrix.
On a related note, we also know that when $P_xP_y=P_yP_x$, then $P_z=P_x+P_y-P_xP_y$. However, I am unable to interpret this condition. Suppose, in a regression context, we have a group of covariates $X=(X_1:X_2)$. What would be the implication of the assumption that the projection matrices corresponding to $X_1$ and $X_2$ commute?
Edit: By projection matrix of $X$, I mean the orthogonal projection matrix on the column space of $X$. Here $X,Y$ are non-random. Columns of $X,Y$ are sub-spaces of $\mathbb{R}^n$.
 A: The inequality is true if and only if
$$
\operatorname{im}(Z)\cap\operatorname{im}(X)^\perp\subseteq\operatorname{im}(Y).\tag{1}
$$
(Remark. The asymmetry of the roles of $X$ and $Y$ above is only apparent. Since both $\operatorname{im}(X)$ and $\operatorname{im}(Y)$ are subspaces of $\operatorname{im}(Z)$, the condition is equivalent to $\operatorname{im}(Z)\cap\operatorname{im}(Y)^\perp\subseteq\operatorname{im}(X)$.)
Suppose $(1)$ does not hold. Then there exists a nonzero vector $z_1\in\operatorname{im}(Z)\cap\operatorname{im}(X)^\perp\setminus\operatorname{im}(Y)$. Hence $P_xz_1=0,\,P_yz_1\ne z_1=P_zz_1$,
$$
\langle z_1,(P_x+P_y-P_z)z_1\rangle=0+\|P_yz_1\|^2-\|z_1\|^2=-\|(I-P_y)z_1\|^2<0
$$
and $P_x+P_y-P_z$ is not positive semidefinite.
Now suppose $(1)$ holds. For any $v\in\mathbb R^n$, let
$$
\begin{aligned}
x_1&:=P_xv\in\operatorname{im}(X),\\
y_1&:=(P_z-P_x)v=P_z(I-P_x)v\in\operatorname{im}(Z)\cap\operatorname{im}(X)^\perp\subseteq\operatorname{im}(X)^\perp\cap\operatorname{im}(Y),\\
w_1&:=(I-P_z)v\in\operatorname{im}(Z)^\perp.\\
\end{aligned}
$$
Then $v=x_1+y_1+w_1$ and
$$
\begin{aligned}
\langle v,P_xv\rangle
&=\langle x_1+y_1+w_1,\, x_1\rangle=\langle x_1,\,x_1\rangle,\\
\\
\langle v,P_yv\rangle
&=\langle x_1+y_1+w_1,\, P_yx_1+y_1\rangle\\
&=\langle x_1,P_yx_1\rangle
+\langle x_1,y_1\rangle
+\langle y_1,P_yx_1\rangle
+\langle y_1,y_1\rangle\\
&=\langle x_1,P_y^2x_1\rangle
+0
+\langle P_yy_1,x_1\rangle
+\langle y_1,y_1\rangle\\
&=\langle P_yx_1,P_yx_1\rangle
+0
+\langle y_1,x_1\rangle
+\langle y_1,y_1\rangle\\
&=\langle P_yx_1,P_yx_1\rangle
+0
+0
+\langle y_1,y_1\rangle\\
&=\langle P_yx_1,P_yx_1\rangle
+\langle y_1,y_1\rangle,\\
\\
\langle v,P_zv\rangle
&=\langle x_1+y_1+w_1,\, x_1+y_1\rangle
=\langle x_1,\,x_1\rangle+\langle y_1,\,y_1\rangle.\\
\end{aligned}
$$
Hence $\langle v,(P_x+P_y-P_z)v\rangle=\langle P_yx_1,P_yx_1\rangle\ge0$. Since $v$ is arbitrary, $P_x+P_y\ge P_z$.
A: We clearly have that $P_z-P_x$ and $P_z-P_y$ are projections and $P_z$ commutes with $P_x$ and $P_y.$
We are going to prove that
$$P_z\le P_x+P_y  \iff P_xP_y=P_yP_x\quad (*)$$
We start with proving the fact that for two orthogonal projections $ P$ and $Q$ we have $$ P\le Q\implies PQ=QP\qquad (**)$$ Indeed, we have $0\le I-Q\le I-P.$ Thus $v=Pv$ implies $v=Qv.$ Hence $QP=PQP$ and $PQ=(QP)^*=PQP.$ Thus $PQ=QP.$
Assume $P_z\le P_x+P_y.$ Then $P_z-P_y\le P_x.$ By $(**)$ we get that $P_x$ and $P_z-P_y$ commute. Hence $P_x$ and $P_y$ commute.
The $\Leftarrow$ direction of $(*)$ follows from the identity $$P_z=P_x+P_y-P_xP_y$$ as the operator $P_xP_y$ is positive.
A concrete example can be made up in the two dimensional space by taking two non orthogonal and not parallel vectors $x$ and $y$ and projections on them. Then $P_x$ and $P_y$ do not commute therefore $P_x+P_y\ge P_z$ fails.
