Determinant of block matrix with singular blocks on the diagonal Let $A$ and $D$ be square matrices, and let $B$ and $C$ be matrices of valid shapes to allow the formation of
$$
M =
\begin{bmatrix}
    A & B \\
    C & D
\end{bmatrix}.
$$
If $\det{A}\neq0$, we may use the Schur complement to express $\det{M}$ in terms of its constituent blocks as
$$
\det{M} = \det{A}\cdot\det(D-CA^{-1}B),
$$
and if $\det{D}\neq0$ we have in a similar fashion that
$$
\det{M} = \det(A-BD^{-1}C)\cdot\det{D}.
$$
My question: Does there exist a similar formula expressing $\det{M}$ in terms of its constituent blocks, that is valid in case $\det{A}=\det{D}=0$?
 A: Without loss of generality, let $A$ be $m$-by-$m$ and $D$ be $n$-by-$n$, with $m\ge n$. Consider
$$
f(t)=\det\left(
\begin{array}{cc}
A+tI_m&B\\
C&D
\end{array}
\right).
$$
Obviously,


*

*$f(t)$ is a polynomial of $t$, for which it is analytic on $\mathbb{R}$;

*$f(0)$ returns the desired determinant;

*$g(t)=\det\left(A+tI_m\right)$ is also a polynomial of $t$, for which it has isolated zeros;

*$g(0)=0$ provided that $A$ is singular, yet since $t=0$ is an isolated zero of $g(t)$, $g(t)\ne 0$ for all $t\in\left(-\delta,\delta\right)\setminus\left\{0\right\}$ for some $\delta>0$.


Now, since $g(t)\ne 0$ on some $\left(-\delta,\delta\right)\setminus\left\{0\right\}$, it follows that $A+tI_m$ is invertible on this domain. Therefore,
$$
f(t)=\det\left(A+tI_m\right)\det\left(D-C\left(A+tI_m\right)^{-1}B\right).
$$
Consequently, the continuity of $f(t)$ yields
$$
f(0)=\lim_{t\to 0}\left(\det\left(A+tI_m\right)\det\left(D-C\left(A+tI_m\right)^{-1}B\right)\right).
$$
Now, let us focus on two distinct cases. First, if $A=O_m$, i.e., $A$ is a zero matrix of order $m$. In this case, the above formula gives
\begin{align}
f(0)&=\lim_{t\to 0}\left(\det\left(tI_m\right)\det\left(D-C\left(tI_m\right)^{-1}B\right)\right)\\
&=\lim_{t\to 0}\left(t^m\det\left(D-\frac{1}{t}CB\right)\right)\\
&=\lim_{t\to 0}\left(t^m\det\left(\frac{1}{t}\left(tD-CB\right)\right)\right)\\
&=\lim_{t\to 0}\left(t^{m-n}\det\left(tD-CB\right)\right).
\end{align}
Recall that $m\ge n$. We therefore obtain


*

*If $m>n$, it is obvious that $f(0)=0$;

*If $m=n$, it follows that $f(0)=\det\left(-CB\right)=\left(-1\right)^n\det\left(CB\right)$.


Second, consider $A\ne O_m$. This case is more complicated, and there is no elegant form for $f(0)$ with only $A$, $B$, $C$, and $D$ involved. However, since $A\ne O_m$, we may perform some elementary operations of the second type, i.e., row- and column-switching transformations, such that after the operations, we obtain
$$
f(0)=\det\left(
\begin{array}{cc}
A'&B'\\
C'&D'
\end{array}
\right),
$$
where, e.g., $A'$ results from $A$ by switching $A$'s rows and columns, such that $A''$, the $k$-by-$k$ square matrix made up of the first $k$ rows and columns of $A'$, is invertible. The existence of such an $A''$ is guaranteed by the fact that $A\ne O_m$. In this way,
$$
f(0)=\det\left(
\begin{array}{cc}
A''&B''\\
C''&D''
\end{array}
\right).
$$
Thanks to the invertibility of $A''$,
$$
f(0)=\det\left(A''\right)\det\left(D''-C''\left(A''\right)^{-1}B''\right).
$$
This result is much less elegant. $A''$ is only part of $A$. $B''$ contains part of both $A$ and $B$, and so does $C''$. $D''$ contains the whole $D$, and part of $A$, $B$, and $C$. Besides, there are switches of rows and columns as well.
A: For $M=\begin{bmatrix}A & B\\C & D\end{bmatrix}$ let $N=M^TM$. So that
$$N:=\begin{bmatrix}E & F\\F^T & G\end{bmatrix}$$
where
$$\begin{align}
E&=A^T A+C^T C\\F&=A^T B+C^T D\\G&=B^T B+D^T D
\end{align}$$
Now, if $M$ is non-singular then $N$ is a positive definite matrix. Hence according to this other post, $E$ is a positive definite (i.e. non-singular) block matrix.
Since $E$ is non-singular, we can use Schur complement to obtain $\det N$.
$$\det{N} = \det{E}\cdot\det(G-F^T E^{-1}F)=(\det M)^2$$
The only remaining part in this case is to determine the sign of $\det M$, which apparently, there is no easy way to do that in general.
We might be able to get the sign of $\det M$ by a trick similar to the one discussed at the end of @hypernova's answer. I'll update this answer if anything comes up.
A: Hint:
$$
M =
\begin{bmatrix}
    A & B \\
    C & D
\end{bmatrix}.
=
\begin{bmatrix}
    0 & I \\
    I & 0
\end{bmatrix}.
\begin{bmatrix}
    C & D \\
    A & B
\end{bmatrix}
$$
Where $I$ is the idenity matrix of appropriate size.
