1
$\begingroup$

Assume that $X = [X1 : X2]$ is of full column rank matrix (X is not necessarily square)

then

$$X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2$$

is it nonsingular(invertible)?

$P_1=X_1(X_1^TX_1)^{-1}X_1^T$(the projection matrix of X1)

rewrite above matrix

$$X_2^T(I-P_1)X_2$$ Why this matrix is invertible?

What property is it related to?

(I saw this question but didn't understand it clearly

When is Block-Partitioned Matrix Invertible?)

$\endgroup$
2
  • $\begingroup$ I think you mean:$ P_1=I-X_1(X_1^TX_1)^{-1}X_1^T$ and rewrite the above matrix as $X_2^T(P_1)X_2$ $\endgroup$ Jun 12, 2020 at 22:50
  • $\begingroup$ sorry~ I corrected it~ $\endgroup$ Jun 13, 2020 at 8:04

1 Answer 1

1
$\begingroup$

The OP seems to tacitly be working over reals without saying so, otherwise e.g. it isn't clear that $(X_1^T X_1)^{-1}$ exists.

0.) $Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$

1.) over reals we have
$\text{rank}\Big(A^TA\Big) =\text{rank}\Big(A\Big)$
and since $X_2^T Q X_2 =X_2^T Q^2 X_2 = X_2^TQ^T Q X_2$ it suffices to compute $\text{rank}\Big(X_2^T Q X_2\Big) = \text{rank}\Big(Q X_2\Big)$ and show that the RHS has full column rank. Equivalently we want to prove
$\text{rank}\big(Q X_2\big) =\text{rank}\big(X_2\big)$

2.) Since the original matrix $\mathbf X$ has all columns linearly independent but may not be square, it becomes convenient to extend this to a basis, resulting in the $\text{n x n}$ matrix

$\mathbf X' := \bigg[\begin{array}{c|c|c}X_1 & X_2 & X_3\end{array}\bigg] =\bigg[\begin{array}{c|c}\mathbf X & X_3\end{array}\bigg]$
such that $\det\big(\mathbf X'\big) \neq 0$

suppose $X_1$ has r columns, then
$n-r= \text{rank}\Big(Q\mathbf X'\Big) = \text{rank}\Big(Q\Big) = \text{trace}\Big(Q\Big)$

And
$Q\mathbf X' = \bigg[\begin{array}{c|c|c}Q X_1 & Q X_2 & Q X_3\end{array}\bigg]= \bigg[\begin{array}{c|c|c}\mathbf 0 & Q X_2 & Q X_3\end{array}\bigg] $

where the Right Hand Side is an $\text{n x n}$ matrix with the first $r$ columns zero'd out and has rank $n-r$ i.e. this implies $\text{rank}\big(QX_2\big) = \text{rank}\big(X_2\big)$ which completes the proof.

$\endgroup$
2
  • $\begingroup$ I corrected the miss formula , but is there any change in the answer? $\endgroup$ Jun 13, 2020 at 8:28
  • $\begingroup$ your new choice of notation breaks with the link you've given. To sidestep all this I created $Q:= I -X_1(X_1^TX_1)^{-1}X_1^T$ and changed the variables in the writeup. $\endgroup$ Jun 13, 2020 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.