Outer Regularity of Outer Measure vs Measurable iff Almost Open

I'm working through Tao's measure theory notes here, and am struggling with the following concept:

Let $$m^*(E)$$ denote the (lebesgue) outer measure of $$E$$, and let $$\varepsilon > 0$$.

We know $$m^*(E) = \inf \limits_{E \subseteq U \text{ open}} m^*(U)$$. So we can find $$U$$ open with $$m^*(U) - m^*(E) < \varepsilon$$.

We also know that $$E$$ is measurable exactly when some $$m^*(U \setminus E) < \varepsilon$$ for some $$U$$ open.

At first glance, this would seem to suggest that every set is measurable - it seems entirely reasonable that we could turn $$m^*(U) - m^*(E)$$ into $$m^*(U \setminus E)$$ somehow... Obviously nonmeasurable sets do exists, and so there must be no such manipulation.

My insticts want me to try to break this with a nonmeasurable set, perhaps a Vitali Set $$V$$, but it isn't clear to me what an open set approximating $$V$$ (in outer measure, obviously) would look like. This is probably because of the weirdness that happens with the axiom of choice, where my intuition fails.

The question, then is "why does this fail?" What about nonmeasurable sets prevents this (seemingly obvious) manipulation from going through?

Edit: Somewhat more precisely, what does an open set $$U$$ with $$m^*(U) - m^*(V) < \varepsilon$$ look like, and why is $$m^*(U \setminus V)$$ not small?

• Your seem to make this mistake of thinking that $m^{*}(U\setminus V)=m^{*}(U)-m^{*}( V)$ which is not true. You cannot make $m^{*}(U\setminus V)$ small using the fact that $m^{*}(U)-m^{*}( V)$ is small. Oct 1, 2019 at 5:41
• @KaviRamaMurthy - I state in the question that I know I have this misconception. Indeed my question is "why can we not make $m^*(U \setminus V)$ small". Oct 1, 2019 at 6:14
You can build a Vitali set (in $$[0,1]$$) with full outer measure, whose complement also has full outer measure. That is, the open set approximating $$V$$ is the entire interval $$(0,1)$$, but $$(0,1) \setminus V$$ also has full measure.