Contrapositive Measure Proof construction I want to prove the following:
Let $m^*(E) < \infty$. Show that if $E$ is not measurable,then there is an open set $\mathcal{O}$ containing $E$ that has finite outer measure and for which
$$m^*(\mathcal{O}\sim E) > m^*(\mathcal{O})-m^*(E).$$
Proof:
Since $E$ is not mensurable, then this implies that  $\exists \epsilon > 0, \forall \mathcal{O} \supset E$ and $m^*(\mathcal{O}\sim E ) \geq \epsilon$. In particular, since $m^*(E) < \infty$, there must be some $\mathcal{O}$ with finite outer measure so that $\forall \epsilon>0,$ we get:
$$m^*(E) + \epsilon > m^*(\mathcal{O})\Rightarrow m^*(\mathcal{O}) - m^*(E) < \epsilon$$
but,
$$m^*(\mathcal{O}\sim E ) \geq \epsilon \Rightarrow m^*(\mathcal{O}) - m^*(E) \leq m^*(\mathcal{O}\sim E ),$$
as required.
 A: In fact a stronger result holds: 

If there is a measurable set $A$ with $m^∗(A)<\infty$ and $E\subseteq A$ with
  $$m^∗(A\sim E) \leq m^∗(A)−m^∗(E)$$
  then $E$ is measurable. 

Proof: Note that, since $m^∗(A)<\infty$ and $E\subseteq A$, we have $m^∗(E)<\infty$. So $E$ has a mensurable cover $D$ (it means $E\subseteq D$, $D$ is measurable  and $m^*(D)=m^*(E)$). 
Let $B=D\cap A$. It is easy to see that $B \subseteq A$ and that $B$ is also a mensurable cover of $E$. In fact, $E\subseteq B$, $B$ is measurable, and $m^*(E)\leq m^*(B)\leq m^*(D)=m^*(E)$, so $m^*(B)=m^*(E)$. 
Since $A \sim E =(A \sim B) \cup (B \sim E)$, using the the sub-additivity of $m^*$, we have
$$ m^*(A \sim E)\leq m^*(A \sim B) + m^*(B \sim E)$$
So, since $m^*(A \sim B)\leq m^*(A) < \infty$ we have 
$$ m^*(B \sim E) \leq m^*(A \sim E)- m^*(A \sim B) $$
By hypothesis, we have that $m^∗(A\sim E) \leq m^∗(A)−m^∗(E)$, and since $m^*(B)=m^*(E)$, we have 
$$ m^*(B \sim E) \leq m^*(A \sim E)- m*(A \sim B) \leq m^∗(A)−m^∗(B) - m^*(A \sim B) $$
But $A$ and $B$ are measurable, and so is $A\sim B$, then we have (using again that $m^*(A) < \infty$): 
$$ m^*(B \sim E) \leq m^∗(A)−m^∗(B) - m^*(A \sim B)= m^∗(A)−m^∗(A)=0$$
So $m^*(B \sim E)=0$, which implies that $B\sim E$ is measurable. Since $E=B \sim (B\sim E)$, we have that $E$ is measurable.
Remark: You are trying to prove 

If $E$ is not measurable, then there is an open set $\mathcal{O}$ containing $E$ that has finite outer measure and for which
  $$m^*(\mathcal{O}\sim E) > m^*(\mathcal{O})-m^*(E).$$ 

If there is, at least, one open set $\mathcal{O}$ containing $E$ that has finite outer measure and is measurable, then the result you are trying to prove is an immediate consequence of the general result I proved above in this answer. 
Remark 2 There are rather "pathological" examples where there is $E$ such that $m^*(E)<+\infty$ and $E$ is NOT measurable, but there is NO open set $\mathcal{O}$ containing $E$ such that $\mathcal{O}$ has finite outer measure and $\mathcal{O}$  is measurable. In such cases, the general result (proved above) remains true, but the result you are trying to prove may be FALSE. 
Here is one such example: Let $X=\{0,1,2\}$,  $\Sigma=\{\emptyset, \{0,1\}, \{2\}, X\}$ and $m$ be the measure defined by $m(\emptyset)=0$, $m(\{0,1\})=1$, $m(\{2\})=\infty$ and $m(X)=\infty$. It is easy to see that $\Sigma$ is a $\sigma$-algebra and that $m$ is a measure. Let $\tau= \{\emptyset, \{1\}, \{2\},\{1,2\},\{0,2\}, X\}$. It easy to see that $\tau$ is a topology on $X$. 
We can prove that:   


*

*$m^*(B)=0$ if and only $B=\emptyset$   

*$m^*(B)=+\infty$ if and only $2 \in B$   

*$m^*(B)=1$ if and if  $B\neq\emptyset$ and $B\subseteq \{0,1\}$


and that the $\mu^*$-measurable sets are precisely those in $\Sigma$. 
Now consider $E=\{1\}$. We have that $m^*(E)=1$, and $E$ is NOT $\mu^*$-measurable. 
Note that for all $\mu^*$-measurable set $A$ with $m^∗(A)<\infty$ and $E\subseteq A$, we have $$m^∗(A\sim E) > m^∗(A)−m^∗(E)$$ Proof: In fact, there is only one  $\mu^*$-measurable set $A$ with $m^∗(A)<\infty$ and $E\subseteq A$,  it is $\{0,1\}$, and it is to check that  $$m^∗(\{0,1\}\sim E) = 1 > 0 = m^∗(\{0,1\})−m^∗(E)$$ So the general result remains true.  
On the other hand, the only open set $\mathcal{O}$ containing $E$ such that $\mathcal{O}$ has finite outer measure is $E$ itself, that is to say, $\mathcal{O} =E$. But, for such $\mathcal{O}$, we have
$$m^∗(\mathcal{O}\sim E)=\mu^*(\emptyset)=0 \ngtr m^∗(\mathcal{O})−m^∗(E)=1-1=0$$ So the  result you are try to prove is not true in this case.
(Note that there is NO open set $\mathcal{O}$ containing $E$ such that $\mathcal{O}$ has finite outer measure and $\mathcal{O}$ is measurable, so, in this case, the  result you are try to prove does not follow from the general result.) 
Summarizing: The result you are trying to prove is NOT true in general. To ensure it is true, we must assume (add a condition) that there is, at least, one open set $\mathcal{O}$ containing $E$ such that $\mathcal{O}$ has finite outer measure and $\mathcal{O}$ is measurable. 
There several other conditions (for instance, the measure being outer regular) which would automatically imply that there is, at least, one open set $\mathcal{O}$ containing $E$ such that $\mathcal{O}$ has finite outer measure and $\mathcal{O}$ is measurable. 
