A convex decomposition of probability measures I recently came across a problem when I read M.Hochman's paper.
He said before Lemma 3.3 that 'it follows from standard measure theory that for every $\epsilon>0$, there is a $\delta>0$ such that if $||\mu-\nu||<\delta$ then there are probability measures $\tau,\mu',\nu'$ such that $\mu=(1-\epsilon)\tau+\epsilon\mu'$ and $\nu=(1-\epsilon)\tau+\epsilon\nu'$'. Here we just consider the Borel probability measures on $\mathbb{R}^d$, and denote $||\mu-\nu||=\sup_{A\in\mathcal{B}(\mathbb{R}^d)}|\mu(A)-\nu(A)|$  the total variation.
I do not know how to prove this assertion and cannot find any material that contains a proof. I believe the key point is to find a probability measure $\tau$ satisfying
$$\mu(A)-(1-\epsilon)\tau(A)\geq0,\quad\nu(A)-(1-\epsilon)\tau(A)\geq0\quad\forall A\in\mathcal{B}(\mathbb{R}^d).$$
But I do not know how to start. Would you  give me some hints or materials? Thank you very much!
 A: I answer the last question only. Let $\lambda := (\mu + \nu)/2$. Then, $\mu \ll \lambda$ and $\nu \ll \lambda$. $\lambda$ is also a probability measure on $\mathbb{R}^d$.
Then there exists the Radon-Nikodym density $f := \dfrac{d\mu}{d\lambda}$ and $g := \dfrac{d\nu}{d\lambda}$. Let $\displaystyle \widetilde{\tau}(A) := \int_A f \wedge g \, d\lambda, \ A \in \mathcal{B}(\mathbb{R}^d)$. Then, $\tau(A) \le \mu(A) \wedge \nu(A), \ A \in \mathcal{B}(\mathbb{R}^d)$. In particular $\widetilde{\tau}(\mathbb{R}^d) \le 1$.
Now we let $\displaystyle \tau(A) := \frac{\widetilde{\tau}(A)}{\widetilde{\tau}(\mathbb{R}^d)}, \ A \in \mathcal{B}(\mathbb{R}^d)$.
This is a probability measure if $\widetilde{\tau}(\mathbb{R}^d) > 0$. If $\widetilde{\tau}(\mathbb{R}^d) = 0$, then, $f \wedge g = 0$ $\lambda$-a.s., and hence "$f=0$ or $g=0$" holds $\lambda$-a.s., and hence $\mu$ and $\nu$ are singular to each other, and furthermore $\|\mu-\nu\| = 1$.
If $\widetilde{\tau}(\mathbb{R}^d) = 1$, then, $f = g = f \wedge g$, $\lambda$-a.s. and hence $\mu \equiv \nu$. Hence we can assume $\widetilde{\tau}(\mathbb{R}^d) < 1$ if $\mu \ne \nu$.
A: There is nothing particular about the measurable space $\big(\mathbb{R}^d,\mathscr{B}(\mathbb{R}^d)\big)$ regarding the decomposition introduced in the OP. As we will see, the decomposition holds in any general measurable space $(X,\mathscr{B})$.
Recall from analysis (the Hahn-Jordan decomposition for example) that for any finite signed measure $m$,
$$|\mu|(X)=\|m\|_{TV}=\sup_{A\in\mathscr{B}}\{m(A)-m(X\setminus A)\}$$
If $m=\mu-\nu$, where $\mu,\nu\in\mathcal{P}(X)$, then
\begin{align}
\|\mu-\nu\|_{TV}&=\sup_{A\in\mathscr{B}}\{\mu(A)-\nu(A)-\big(1-\mu(A)-(1-\nu(A))\big)\}\\
&=2\sup_{A\in\mathscr{B}}\{\mu(A)-\nu(A)\}=2\|\mu-\nu\|_{PrTV}
\end{align}
where the sup is taken over measurable sets and $PrTV$ stands for total variation as used in probability. Recall that for any finite signed measures $\mu$ and $\nu$,
\begin{align}
\mu-\nu&=(\mu-\nu)_+-(\mu-\nu)_-\\
|\mu-\nu|&=(\mu-\nu)_+ + (\mu-\nu)_+
\end{align}
If $\mu$ and $\nu$ are probability measures it follows that
$$\|(\mu-\nu)_+\|_{TV}=(\mu-\nu)_+(X)=(\mu-\nu)_-(X)=\|(\mu-\nu)_-\|_{TV}$$
Frome here on, $\mu$ and $\nu$ are probability measures.
Assume w.l.g. that there is a probability measure $\eta$ such that $\mu\ll\eta$ and $\nu\ll\eta$. Then there are nonnegative measurable functions $f$ and $g$ such that $\mu=f\cdot \eta$ and $\nu=g\cdot\eta$.
Given $0<\varepsilon<1$, define $\delta=\min(\varepsilon,1/2)$.
Suppose $f\neq g$. Then $0< c=\int f\wedge g\,d\eta<1$, and
\begin{align}
1-c&=\int f-f\wedge g \,d\eta=\int (f-g)_+\,d\eta=\int(f-g)_-\,d\eta\\
&=\int(g-f)_+\,d\eta=\int g-f\wedge g\,d\eta=\|\mu-\nu\|_{PrTV}<\delta
\end{align}
and so $\frac{1-\varepsilon}{c}<1$. It follows that
$$f-(1-\varepsilon)c^{-1}(f\wedge g)=\Big(\big(1-\frac{1-\varepsilon}{c}\big)f\Big)\vee \Big(f-\frac{1-\varepsilon}{c}g\Big)=:h\geq0$$
and  $\int h\,d\eta=\varepsilon$. Setting $\tau=c^{-1}(f\wedge g)\cdot\eta$, and $\mu'=\varepsilon^{-1}h\cdot\eta$ yields the desire decomposition for $\mu$:
$$\mu=(1-\varepsilon)\tau+\varepsilon \mu'$$
Similarly,
$$g-(1-\varepsilon)c^{-1}(f\wedge g)=\Big(\big(1-\frac{1-\varepsilon}{c}\big)g\Big)\vee \Big(g-\frac{1-\varepsilon}{c}f\Big)=:\ell\geq0$$
and so, $\int\ell\,d\eta=\varepsilon$. Setting $\nu'=\ell\cdot\eta$ yields the desired decomposition for $\nu$:
$$\nu=(1-\varepsilon)\tau+\varepsilon \nu'$$
