Inverse of a $2 \times 2$ block matrix Let
$$S := \pmatrix{A&B\\C&D}$$
If $A^{-1}$ or $D^{-1}$ exist, we know that matrix $S$ can be inverted.
$$S^{-1} = \pmatrix{A^{-1}+A^{-1}B(D-CA^{-1}B)^{-1}CA^{-1}&-A^{-1}B(D-CA^{-1}B)^{-1}\\-(D-CA^{-1}B)^{-1}CA^{-1}&(D-CA^{-1}B)^{-1}}$$
But, what if $A^{-1}$ and $D^{-1}$ do not exist? Can we invert matrix $S$?
For example,
$$S = \pmatrix{0&1\\1&0}$$
or
$$S = \pmatrix{2&3&1&1\\4&6&1&2\\1&1&3&1\\4&1&12&4}$$
both their $A^{-1}$ and $D^{-1}$ don't exist, but $S^{-1}$ exists.
 A: If your block matrix is real or complex and known to be invertible, you may apply the usual block matrix inverse formula to find $(S^\ast S)^{-1}$ and thus to calculate $S^{-1}=(S^\ast S)^{-1}S^\ast$. See my answer to another question for more details.
A: As the comments discuss, the claim that $S$ is invertible if $A$ is invertible as claimed in the question is false. What is true is that:

Theorem: If $A$ is invertible, then $S$ is invertible if and only if the Schur complement $D-CA^{-1}B$ is invertible. In which case, the stated formula for the inverse of $S$ in $2\times 2$ block form holds.

So $A$ does not need to be invertible for $S$ to be invertible, but if $A$ is, then $D-CA^{-1}B$ must be invertible as well for $S$ to be invertible.
As the OP notes with their example $\begin{bmatrix}0&1\\1&0\end{bmatrix}$, the top-left matrix need not be invertible for $S$ to be invertible. Instead, for $S$ to be invertible, it just has to be possible for one to reorder the rows and columns of the matrix $S$ (left- and right-multiplying by permutation matrices) to bring a nonsingular matrix $A$ into the block $(1,1)$-position and the resulting reordered Schur complement $D-CA^{-1}B$ to be nonsingular. Indeed, the additional cases in Elias’ answer all boil down to reordering the rows and columns of $S$ to put either $B$, $C$, or $D$ into the top-left spot and then applying the theorem.
An important observation is that one only needs to reorder the rows of $S$ to check invertibility of $S$, the core observation behind LU factorization with partial pivoting:

Theorem: Fix $k$ between $1$ and the size of $S$, $S$ is invertible if and only if it is possible to find a permutation matrix $P$ where $PS$ has block form $PS=\begin{bmatrix} A&B\\C&D\end{bmatrix}$, with $A$ being $k\times k$ and $A$ and $D-CA^{-1}B$ are invertible.

A: I saw a paper on this topic by Lu and Shiou some years ago. Here is the link. They first introduced the formula you mentioned and then investigated other special cases.
A: One good reference is the Handbook Matrix Mathematics Theory, Facts and Formulas by Dennis S. Bernstein. Let $\mathbb{F}$ equal to $\mathbb{R}$ or  $\mathbb{C}$.

Proposition 3.9.7. Let $A \in \mathbb{F}^{n \times n}, B \in \mathbb{F}^{n \times m}, C \in \mathbb{F}^{m \times n}$, and $D \in \mathbb{P}^{m \times m}$. If $A$ and $D-C A^{-1} B$ are nonsingular, then
$$
\left[\begin{array}{ll}
A & B \\
C & D
\end{array}\right]^{-1}=\left[\begin{array}{rr}
A^{-1}+A^{-1} B\left(D-C A^{-1} B\right)^{-1} C A^{-1} & -A^{-1} B\left(D-C A^{-1} B\right)^{-1} \\
-\left(D-C A^{-1} B\right)^{-1} C A^{-1} & \left(D-C A^{-1} B\right)^{-1}
\end{array}\right]
$$

There are three more cases to consider in addition to the above proposition.

*

*If $D$ and $A-B D^{-1} C$ are nonsingular;

*If $B$ and $C-D B^{-1} A$ are nonsingular;

*If $C$ and $B-A C^{-1} D$ are nonsingular;

For the sake of illustration we will prove only the case where $C$ and $B-A C^{-1} D$ are nonsingular;
To prove these cases it is enough to use the matrix $J=\left[\begin{array}{cc}  0 & I \\ I & 0 \end{array} \right]$ to reduce them to the same case as the proposition above. Observe that
$
\left[\begin{array}{cc}  0 & I \\ I & 0 \end{array} \right]^{-1}
=
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\mbox{ once } 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\cdot 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
=
\left[ \begin{array}{cc} I & 0 \\ 0 & I \end{array} \right].
$
Another important thing to note is that
$$
\;(\ast)\;\;\qquad \left[ \begin{array}{cc} U & V \\ X & Y \end{array} \right]
=
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\cdot 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\cdot
\left[ \begin{array}{cc} U & V \\ X & Y \end{array} \right]
=
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\cdot
\left[ \begin{array}{cc} X & Y \\ U & V \end{array} \right].
$$
and
$$
(\ast\ast)\qquad \left[ \begin{array}{cc} U & V \\ X & Y \end{array} \right]
=
\left[ \begin{array}{cc} U & V \\ X & Y \end{array} \right]
\cdot 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\cdot 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
=
\left[ \begin{array}{cc} V & U \\ Y & X \end{array} \right]
\cdot 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
$$
Suppose there exists the inverse $C^{-1}$ of $C$ and there exists the inverse $(B -AC^{-1}D)^{-1}$ of $(B -AC^{-1}D)$.  What can we say about the inverse of $\left[ \begin{array}{cc} C & D \\ A & B \end{array} \right]$? By proposition above we have
$$
\left[ \begin{array}{cc} C & D \\ A & B \end{array} \right]^{-1}
=\left[\begin{array}{rr}
 C^{-1}+C^{-1} D\left(B-A C^{-1} D\right)^{-1} A C^{-1} & -C^{-1} D\left(B-A C^{-1} D\right)^{-1}  \\
-\left(B-A C^{-1} D\right)^{-1} A C^{-1} & \left(B-A C^{-1} D\right)^{-1} 
\end{array}\right]
$$
And more, by $(\ast)$ and $(\ast\ast)$ we have
\begin{align}
\left[ \begin{array}{cc} A & B \\ C & D \end{array} \right]^{-1}
&=
\left( 
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\cdot
\left[ \begin{array}{cc} C & D \\ A & B \end{array} \right]
\right)^{-1}
\\
&=
\left[ \begin{array}{cc} C & D \\ A & B \end{array} \right]^{-1}
\cdot
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]^{-1}
\\
&=
\left[ \begin{array}{cc} C & D \\ A & B \end{array} \right]^{-1}
\cdot
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\\
&=
\left[\begin{array}{rr}
 C^{-1}+C^{-1} D\left(B-A C^{-1} D\right)^{-1} A C^{-1} & -C^{-1} D\left(B-A C^{-1} D\right)^{-1}  \\
-\left(B-A C^{-1} D\right)^{-1} A C^{-1} & \left(B-A C^{-1} D\right)^{-1} 
\end{array}\right]
\cdot
\left[ \begin{array}{cc} 0 & I \\ I & 0 \end{array} \right]
\\
&=
\left[\begin{array}{rr}
-C^{-1} D\left(B-A C^{-1} D\right)^{-1} & C^{-1}+C^{-1} D\left(B-A C^{-1} D\right)^{-1} A C^{-1} \\
\left(B-A C^{-1} D\right)^{-1} & -\left(B-A C^{-1} D\right)^{-1} A C^{-1}
\end{array}\right]
\end{align}
