let $P,Q$ be two probability measures on $(\Omega, \mathscr {F})$, and let $\nu$ be a $\sigma$-finite measure on the same event space such that $P \ll v, Q \ll v$. Define $\frac{dP}{dv}=p$, $\frac{dQ}{dv}=q$.

The total variation distance between P and Q is then: $$ V(P,Q) = \sup_{A \in \mathscr{F}}|P(A) - Q(A)|= \sup_A \bigg| \int_A(p-q )d\nu \bigg| $$ I'm confused about the following, we may write: $$ V(P,Q) = \frac{1}{2} \int |p-q|d\nu $$ First, how can we bring the absolute value inside the integral and get rid of the supremum, and second, what region are we integrating over now? This statement is made frequently in the book: Introduction to nonparametric Estimation - Tsybakov


1 Answer 1


Let $B = \{p \ge q\}$. Note that \begin{align*} \int_\Omega \def\abs#1{\left|#1\right|}\abs{p-q}\, d\nu &= \int_B (p - q) \, d\nu + \int_{\Omega \setminus B} (q- p)\, d\nu\\ &\le 2 \sup_A \abs{\int_A (p-q) \, d\nu} \end{align*} On the other side, note first that $$ \int_\Omega (p-q) \,d\nu = P(\Omega) - Q(\Omega) = 0 $$ and hence $$ \int_B (p-q) \, d\nu = \int_{\Omega \setminus B} (q-p) \, d\nu $$ Now for any $A \in \mathscr F$, we have \begin{align*} \abs{\int_A (p-q)\, d\nu} &= \max\left\{\int_A (p-q)\, d\nu, \int_A (q-p)\, d\nu\right\}\\ &\le\max\left\{ \int_{A\cap B} (p-q)\, d\nu, \int_{A \cap (\Omega \setminus B)} (q-p)\, d\nu\right\}\\ &\le \max\left\{ \int_{B} (p-q)\, d\nu, \int_{\Omega \setminus B} (q-p)\, d\nu\right\}\\ &= \int_B (p-q)\, d\nu\\ &= \frac 12 \int_\Omega \abs{p-q}\,d\nu \end{align*} Taking the supremum over $A \in \mathscr F$, gives $$ \sup_A \abs{\int_A (p-q)\, d\nu} \le \frac 12 \int_\Omega \abs{p-q}\, d\nu $$ which is the other needed inequality.

  • $\begingroup$ why does $ \int_B (p-q)\, d\nu = \frac 12 \int_\Omega \abs{p-q}\,d\nu$ ? $\endgroup$
    – Monolite
    Jul 10, 2018 at 1:23
  • $\begingroup$ This is because $$\int_B (p-q) d\nu + \int_{\Omega\backslash B } (p-q) d\nu = \int_\Omega(p-q) d\nu= 0$$, thus $\int_B (p-q) d\nu = - \int_{\Omega\backslash B } (p-q) d\nu$. However $p\geq q$ on $B$ and $p<q$ on the complement of $B$, meaning that $$\int_B |p-q| d\nu=-\int_{\Omega\backslash B } (p-q) d\nu = \int_{\Omega\backslash B } |p-q| d\nu$$ The leftmost and rightmost integrals are the same and sum to the integral $I=\int_\Omega |p-q|d\nu$, so they must be half of $I$. $\endgroup$
    – wilsonw
    Aug 5, 2020 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.