Total variation distance is a measure for comparing two probability distributions (assuming that these are unit vectors in a finite space- where basis corresponds to the sample space ($\omega$)). I know a distance measure need to obey triangle inequality and it should satisfy that orthogonal vectors have maximum distance and the same distributions should have distance $0$. Others should like between these two. I completely don't understand why the $L^1$ norm is chosen for measuring the distance between these vectors (prob. distributions). I also want to know why it is exactly defined the way it is. $TV(P_1,P_2) = \frac{1}{2}\sum_{x \in \omega} \mid {P_1(x)-P_2(x) \mid}$

  • $\begingroup$ Yes. That's what but here $f,g$ are probability distributions. I don't understand why $L^1$ is used and why 1/2 comes in the way? $\endgroup$ – am_rf24 Oct 30 at 20:20
  • $\begingroup$ The total variation distance of two probability measures is usually defined in terms of the sup norm: en.wikipedia.org/wiki/… $\endgroup$ – amsmath Oct 30 at 20:22
  • $\begingroup$ "When the set($\omega$) is countable, the total variation distance is related to the $L^1$ norm by the identity." So, this follows by doing some math. But what is the intuitive explanation for this? $\endgroup$ – am_rf24 Oct 30 at 20:36
  • $\begingroup$ See Proposition 4.2 in pages.uoregon.edu/dlevin/MARKOV/markovmixing.pdf $\endgroup$ – amsmath Oct 30 at 20:42

The two observations for characterizing the total variation distance using the two definitions are $\\$: $ 1. \max{P_1(A)-P_2(A)} = \max{P_2(\omega-A)-P_1(\omega-A)} \\$ $2. P_1(\omega)= P_2(\omega) \implies P_1(A)- P_2(A) = P_2(B) -P_1(B) = \max_{A \subset \omega}( P_1(A)- P_2(A)) $

Now, using $ 1/2 \sum_{x}{\mid{P_1(x)-P_2(x)}\mid}= \frac{1}{2}\sum_{x:P_1(x)>P_2(x)}{{(P_1(x)-P_2(x))}}+ \frac{1}{2}\sum_{x:P_2(x)>P_1(x)}{{(P_2(x)-P_1(x))}}$ $ \qquad \qquad \qquad \qquad= \frac{1}{2}\max_{A \subset \omega}{(P_1(A)-P_2(A))} + \frac{1}{2} \max_{B \subset \omega}{(P_2(B)-P_1(B))}$
$ \qquad \qquad \qquad \qquad= \max_{A \subset \omega}{ \mid P_1(A)-P_2(A)\mid}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.