Suppose I have a block partitioned matrix \begin{equation} \begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & \mathbf{X}_2^{\top}\mathbf{X}_2 \\ \end{bmatrix} \end{equation} where $\mathbf{X}_1$ is $G \times K_1$ and $\mathbf{X}_2$ is $G \times K_2$. Thus our block partitioned matrix is equal to the outer product $\begin{bmatrix} \mathbf{X}_1^{\top} \\ \mathbf{X}_2^{\top} \end{bmatrix}\begin{bmatrix} \mathbf{X}_1 & \mathbf{X}_2 \end{bmatrix}$ and is dimension $K \times K$ where $K=K_1+K_2$.

In general, when is this block-partitioned matrix invertible? Is there a necessary and sufficient condition?

Edit. My first thought is: for the block-partitioned matrix to be invertible, it is equivalent that each of the four blocks are invertible. However, I no longer think this. For example, if one of the off diagonals, say the lower left, $\mathbf{X}_2^{\top}\mathbf{X}_1=\mathbf{0}$, the matrix could still be invertible if the blocks on the diagonal are. I can't articulate a proof of that though -- could anyone help?



The matrix $$ Y:=X^TX, \quad X:=[X_1,X_2], $$ (which is generally positive semidefinite) is invertible iff $[X_1,X_2]$ has full column rank.

So necessarily, $X_1$ must have full column rank. However, full rank of $X_2$ is not sufficient for the nonsingularity of $Y$. From the block inversion formulas it follows that

$X$ is invertible iff $X_1$ has full column rank and the Schur complement $$\tag{1}X_2^TX_2-X_2^TX_1(X_1^TX_1)^{-1}X_1^TX_2=X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2$$ is invertible.

The matrix $P_1:=I-X_1(X_1^TX_1)^{-1}X_1^T$ is an orthogonal projector onto the complement of the range of $X_1$, that is, the nullspace of $X_1^T$. Since $X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2=(P_1X_1)^T(P_1X_1)$, we have that (1) is invertible iff $P_1X_1$ has full column rank, that is, the columns of $X_1$ do not become dependent when projected onto the nullspace of $X_1^T$.

Note that the same argument can be made with the indices $1$ and $2$ exchanged.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.