1
$\begingroup$

Assume that $X = [X1 : X2]$ is of full column rank matrix (X is not necessarily square)

then

$$X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2$$

is it nonsingular(invertible)?

$P_1=X_1(X_1^TX_1)^{-1}X_1^T$(the projection matrix of X1)

rewrite above matrix

$$X_2^T(I-P_1)X_2$$ Why this matrix is invertible?

What property is it related to?

(I saw this question but didn't understand it clearly

When is Block-Partitioned Matrix Invertible?)

$\endgroup$
2
  • $\begingroup$ I think you mean:$ P_1=I-X_1(X_1^TX_1)^{-1}X_1^T$ and rewrite the above matrix as $X_2^T(P_1)X_2$ $\endgroup$ Jun 12, 2020 at 22:50
  • $\begingroup$ sorry~ I corrected it~ $\endgroup$ Jun 13, 2020 at 8:04

1 Answer 1

1
$\begingroup$

The OP seems to tacitly be working over reals without saying so, otherwise e.g. it isn't clear that $(X_1^T X_1)^{-1}$ exists.

0.) $Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$

1.) over reals we have
$\text{rank}\Big(A^TA\Big) =\text{rank}\Big(A\Big)$
and since $X_2^T Q X_2 =X_2^T Q^2 X_2 = X_2^TQ^T Q X_2$ it suffices to compute $\text{rank}\Big(X_2^T Q X_2\Big) = \text{rank}\Big(Q X_2\Big)$ and show that the RHS has full column rank. Equivalently we want to prove
$\text{rank}\big(Q X_2\big) =\text{rank}\big(X_2\big)$

2.) Since the original matrix $\mathbf X$ has all columns linearly independent but may not be square, it becomes convenient to extend this to a basis, resulting in the $\text{n x n}$ matrix

$\mathbf X' := \bigg[\begin{array}{c|c|c}X_1 & X_2 & X_3\end{array}\bigg] =\bigg[\begin{array}{c|c}\mathbf X & X_3\end{array}\bigg]$
such that $\det\big(\mathbf X'\big) \neq 0$

suppose $X_1$ has r columns, then
$n-r= \text{rank}\Big(Q\mathbf X'\Big) = \text{rank}\Big(Q\Big) = \text{trace}\Big(Q\Big)$

And
$Q\mathbf X' = \bigg[\begin{array}{c|c|c}Q X_1 & Q X_2 & Q X_3\end{array}\bigg]= \bigg[\begin{array}{c|c|c}\mathbf 0 & Q X_2 & Q X_3\end{array}\bigg] $

where the Right Hand Side is an $\text{n x n}$ matrix with the first $r$ columns zero'd out and has rank $n-r$ i.e. this implies $\text{rank}\big(QX_2\big) = \text{rank}\big(X_2\big)$ which completes the proof.

$\endgroup$
2
  • $\begingroup$ I corrected the miss formula , but is there any change in the answer? $\endgroup$ Jun 13, 2020 at 8:28
  • $\begingroup$ your new choice of notation breaks with the link you've given. To sidestep all this I created $Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$ and changed the variables in the writeup. $\endgroup$ Jun 13, 2020 at 18:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .