Compact subset of a Hausdorff space is closed.

I'm having a bit of difficulty understanding the proof that in a Hausdorff space, a compact subset is closed. The proof I'm looking at uses the fact that a finite intersection of open sets is open (e.g. here and here).

While I generally understand what's going on here, I can't quite see why it is important to take the (finite) intersection of open sets in the complement. How is it that we are not interested in the union, which I'm thinking of as the entirety of the complement? Also, how is it that taking any arbitrary open ball in the complement is insufficient?

You’ve misunderstood the basic idea of the argument. I’ll refer to the answer at your first link for notation. We show that $K$ is closed by showing that if $x$ is any point of $\Bbb X$ not in $K$, then $x$ has an open nbhd disjoint from $K$; this is one of the most basic ways of showing that a set is closed. If you’re more used to thinking of closed sets as the complements of open sets, think of it this way: if for each $x\in\Bbb X\setminus K$ we can find an open set $W_x$ such that $x\in W_x\subseteq\Bbb X\setminus K$, then $\bigcup_{x\in\Bbb X\setminus K}W_x$ is an open set whose complement is exactly $K$.

We need to take a finite intersection of open sets in order to get a nbhd of $x$ that is disjoint from all of $K$. Remember, the only separation that we have to start with is Hausdorffness, so for each $y\in K$ we can find disjoint open sets $U_y$ and $V_y$ such that $x\in U_y$ and $y\in V_y$. Since $\{V_y:y\in K\}$ is an open cover of $K$, and $K$ is compact, we can find a finite $F\subseteq K$ such that $\{V_y:y\in F\}$ covers $K$. The individual sets $V_y$ with $y\in F$ may not cover $K$, but their union definitely does:

$$K\subseteq\bigcup_{y\in F}V_y\;.$$

Each $U_y$ for $y\in F$ is disjoint from the corresponding $V_y$, but there’s no reason to think that it’s disjoint from all of them; in particular, it need not be disjoint from $K$. However, suppose that

$$z\in\bigcap_{y\in F}U_y\;;$$

then for each $y\in F$ we have $z\in U_y$, and $U_y\cap V_y=\varnothing$, so $z\notin V_y$. Thus,

$$z\notin\bigcup_{y\in F}U_y\;,$$

and since that union contains $K$, $z\notin K$. In other words, the set $\bigcap_{y\in F}U_y$ is disjoint from $K$. It also contains $x$, since each $V_y$ is a nbhd of $x$, and it’s open, because it’s the intersection of only finitely many open sets. Therefore it’s an open nbhd of $x$ disjoint from $K$. And since $x$ was an arbitrary point of $\Bbb X\setminus K$, no point of $\Bbb X\setminus K$ is a limit point of $K$, and $K$ is therefore closed.

you take the intersection of the opens to make sure that the neighbourhood of the point in the complementary is disjoint from the compact.

Let $k \in K$ and $x \in K^c$. For each $k$ you define two disjoints opens $V_k$ and $U_{k,x}$. By compactness you cover $K$ with a finite number of $V_k$'s. Then define a neighbourhood for $x$ in this way $\bigcap_{i=1}^nU_{k,x}$. Finite intersect is open and it is disjoint form $K$. In fact each point in the new neighbourhood is contained in ALL $U_{k,x}$ and so by definition is disjoint from ALL $V_k$ and hence from $K$. If you take the union instead you have to substitute ALL with AT LEAST ONE and hence you don't have the conclusion.