Why for a compact metric probability space, any Borel subset can be approximated by compact set? Let $X$ be a compact metric space with a Probability Borel measure $\mu$.
Let $C$ be any Borel subset of $X$.
Then for any small positive number $a$, we can find compact set $K$ such that $K$ is subset of C and $\mu(C\setminus K)<a$.
Why is it so?
 A: You can prove this, it is not that difficult if you know what to do. It is a typical $\sigma$-algebra argument. 
Define
$$\tag{1}\mathcal{A}=\left\{ C\subset \Omega | \forall\, \varepsilon,\ \exists U\ \text{open and}\ K\ \text{compact s.t.}\ K\subset C\subset U\ \text{and}\ \mu(U\setminus K)<\varepsilon\right\}.$$
Then you can see that any closed subset $F$ of $\Omega$ lies in $\mathcal{A}$. Indeed, fix $\varepsilon >0$. Define the $\delta$-neighborhood(*) of $F$ to be 
$$F_\delta=\{ x \in \Omega\ |\ \text{dist}(x, \Omega)<\delta \}.$$
This set is open. Since the sequence $F_1, F_{1/2}, F_{1/3}\ldots F_{1/n}\ldots$ is shrinking, by the theorem of continuity of measure you have 
$$\lim_{n \to \infty} \mu(F_{\frac{1}{n}})=\mu\left(\bigcap_{n=1}^\infty F_{\frac{1}{n}}\right)=\mu(F),$$
so there is a $N$ so big that $\mu(F_{1/N})-\mu(F) < \varepsilon$. Then $F$ complies with the definition of $\mathcal{A}$ given in equation (1) with $K=F$ and $U=F_{1/N}$: since $\varepsilon$ was arbitrary this shows that $F\in \mathcal{A}$.
Later you show that  $\mathcal{A}$ is a $\sigma$-algebra and since it contains all closed sets it contains all Borel sets too. 

(*)Which is sometimes called "Minkowski sausage", a name which I find funny.
A: Any Borel probability measure on a metric space is regular, and the condition you describe is precisely regularity of mu since your space is compact.
See: http://en.wikipedia.org/wiki/Regular_measure 
