This comes from an exercise from Real Analysis by Folland.
Let $\mathcal{A}\subset P(X)$ be an algebra, $\mathcal{A}_\sigma$ the collection of countable unions of sets in $\mathcal{A}$, and $\mathcal{A}_{\sigma\delta}$ the collection of countable intersections of sets in $\mathcal{A}_\sigma$. Let $\mu_{0}$ be a premeasure on $\mathcal{A}$ and $\mu^*$ the induced outer measure.
a.) For any $E\subset X$ and $\epsilon > 0$ there exists $A\in \mathcal{A}_\sigma$ with $E\subset A$ and $\mu^*(A) \leq \mu^*(E) + \epsilon$.
b.) If $\mu^*(E) < \infty$, then $E$ is $\mu^*$-measurable if and only if there exists $B\in \mathcal{A}_{\sigma\delta}$ with $E\subset B$ and $\mu^*(B\setminus E) = 0$
proof of a.) was done in an earlier post: Outer measure problem
proof of b.) Using part a.) $\forall \epsilon_i$ there exists $A_i\in \mathcal{A}_\sigma$ such that $E\subset A_i$. Let $B = \bigcap_{i=1}^{n}A_i$ then $B\in\mathcal{A}_{\sigma\delta}$. Now, we need to show that $\mu^*(B\cap E^c) = 0$. From the definition of outer measure we know that: $$\mu^*(B\cap E^c) = \inf\left\{\sum_{1}^{\infty}\mu^*(E_j):E_j\in\mathcal A \ \ \text{and} \ \ (B\cap E^c)\subset \bigcup_{1}^{\infty}E_j\right\}$$ From part a.) we know that $\mu^*(A)\leq \mu^*(E) + \epsilon$. I am not sure if this fact helps and I am stuck where to go from here, any suggestions would be great. Also, any hints on how to prove the converse would be great as well.