Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
Assume that $X = [X1 : X2]$ is of full column rank matrix (X is not necessarily square)
then
$$X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2$$
is it nonsingular(invertible)?
$P_1=X_1(X_1^TX_1)^{-1}X_1^T$(the projection matrix of X1)
rewrite above matrix
$$X_2^T(I-P_1)X_2$$ Why this matrix is invertible?
What property is it related to?
(I saw this question but didn't understand it clearly
The OP seems to tacitly be working over reals without saying so, otherwise e.g. it isn't clear that $(X_1^T X_1)^{-1}$ exists.
0.) $Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$
1.) over reals we have
$\text{rank}\Big(A^TA\Big) =\text{rank}\Big(A\Big)$
and since $X_2^T Q X_2 =X_2^T Q^2 X_2 = X_2^TQ^T Q X_2$ it suffices to compute
$\text{rank}\Big(X_2^T Q X_2\Big) = \text{rank}\Big(Q X_2\Big)$ and show that the RHS has full column rank. Equivalently we want to prove
$\text{rank}\big(Q X_2\big) =\text{rank}\big(X_2\big)$
2.) Since the original matrix $\mathbf X$ has all columns linearly independent but may not be square, it becomes convenient to extend this to a basis, resulting in the $\text{n x n}$ matrix
$\mathbf X' := \bigg[\begin{array}{c|c|c}X_1 & X_2 & X_3\end{array}\bigg] =\bigg[\begin{array}{c|c}\mathbf X & X_3\end{array}\bigg]$
such that $\det\big(\mathbf X'\big) \neq 0$
suppose $X_1$ has r columns, then
$n-r= \text{rank}\Big(Q\mathbf X'\Big) = \text{rank}\Big(Q\Big) = \text{trace}\Big(Q\Big)$
And
$Q\mathbf X' = \bigg[\begin{array}{c|c|c}Q X_1 & Q X_2 & Q X_3\end{array}\bigg]= \bigg[\begin{array}{c|c|c}\mathbf 0 & Q X_2 & Q X_3\end{array}\bigg] $
where the Right Hand Side is an $\text{n x n}$ matrix with the first $r$ columns zero'd out and has rank $n-r$ i.e. this implies $\text{rank}\big(QX_2\big) = \text{rank}\big(X_2\big)$ which completes the proof.
