# Why is this matrix invertible?(nonsingular, full column rank)

Assume that $$X = [X1 : X2]$$ is of full column rank matrix (X is not necessarily square)

then

$$X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2$$

is it nonsingular(invertible)?

$$P_1=X_1(X_1^TX_1)^{-1}X_1^T$$(the projection matrix of X1)

rewrite above matrix

$$X_2^T(I-P_1)X_2$$ Why this matrix is invertible?

What property is it related to?

(I saw this question but didn't understand it clearly

• I think you mean:$P_1=I-X_1(X_1^TX_1)^{-1}X_1^T$ and rewrite the above matrix as $X_2^T(P_1)X_2$ Jun 12, 2020 at 22:50
• sorry~ I corrected it~ Jun 13, 2020 at 8:04

The OP seems to tacitly be working over reals without saying so, otherwise e.g. it isn't clear that $$(X_1^T X_1)^{-1}$$ exists.

0.) $$Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$$

1.) over reals we have
$$\text{rank}\Big(A^TA\Big) =\text{rank}\Big(A\Big)$$
and since $$X_2^T Q X_2 =X_2^T Q^2 X_2 = X_2^TQ^T Q X_2$$ it suffices to compute $$\text{rank}\Big(X_2^T Q X_2\Big) = \text{rank}\Big(Q X_2\Big)$$ and show that the RHS has full column rank. Equivalently we want to prove
$$\text{rank}\big(Q X_2\big) =\text{rank}\big(X_2\big)$$

2.) Since the original matrix $$\mathbf X$$ has all columns linearly independent but may not be square, it becomes convenient to extend this to a basis, resulting in the $$\text{n x n}$$ matrix

$$\mathbf X' := \bigg[\begin{array}{c|c|c}X_1 & X_2 & X_3\end{array}\bigg] =\bigg[\begin{array}{c|c}\mathbf X & X_3\end{array}\bigg]$$
such that $$\det\big(\mathbf X'\big) \neq 0$$

suppose $$X_1$$ has r columns, then
$$n-r= \text{rank}\Big(Q\mathbf X'\Big) = \text{rank}\Big(Q\Big) = \text{trace}\Big(Q\Big)$$

And
$$Q\mathbf X' = \bigg[\begin{array}{c|c|c}Q X_1 & Q X_2 & Q X_3\end{array}\bigg]= \bigg[\begin{array}{c|c|c}\mathbf 0 & Q X_2 & Q X_3\end{array}\bigg]$$

where the Right Hand Side is an $$\text{n x n}$$ matrix with the first $$r$$ columns zero'd out and has rank $$n-r$$ i.e. this implies $$\text{rank}\big(QX_2\big) = \text{rank}\big(X_2\big)$$ which completes the proof.

• I corrected the miss formula , but is there any change in the answer? Jun 13, 2020 at 8:28
• your new choice of notation breaks with the link you've given. To sidestep all this I created $Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$ and changed the variables in the writeup. Jun 13, 2020 at 18:37