# Total variation distance, $L^1$ norm

Total variation distance is a measure for comparing two probability distributions (assuming that these are unit vectors in a finite space- where basis corresponds to the sample space ($$\omega$$)). I know a distance measure need to obey triangle inequality and it should satisfy that orthogonal vectors have maximum distance and the same distributions should have distance $$0$$. Others should like between these two. I completely don't understand why the $$L^1$$ norm is chosen for measuring the distance between these vectors (prob. distributions). I also want to know why it is exactly defined the way it is. $$TV(P_1,P_2) = \frac{1}{2}\sum_{x \in \omega} \mid {P_1(x)-P_2(x) \mid}$$

• Yes. That's what but here $f,g$ are probability distributions. I don't understand why $L^1$ is used and why 1/2 comes in the way? – am_rf24 Oct 30 at 20:20
• The total variation distance of two probability measures is usually defined in terms of the sup norm: en.wikipedia.org/wiki/… – amsmath Oct 30 at 20:22
• "When the set($\omega$) is countable, the total variation distance is related to the $L^1$ norm by the identity." So, this follows by doing some math. But what is the intuitive explanation for this? – am_rf24 Oct 30 at 20:36
• See Proposition 4.2 in pages.uoregon.edu/dlevin/MARKOV/markovmixing.pdf – amsmath Oct 30 at 20:42

The two observations for characterizing the total variation distance using the two definitions are $$\\$$: $$1. \max{P_1(A)-P_2(A)} = \max{P_2(\omega-A)-P_1(\omega-A)} \\$$ $$2. P_1(\omega)= P_2(\omega) \implies P_1(A)- P_2(A) = P_2(B) -P_1(B) = \max_{A \subset \omega}( P_1(A)- P_2(A))$$
Now, using $$1/2 \sum_{x}{\mid{P_1(x)-P_2(x)}\mid}= \frac{1}{2}\sum_{x:P_1(x)>P_2(x)}{{(P_1(x)-P_2(x))}}+ \frac{1}{2}\sum_{x:P_2(x)>P_1(x)}{{(P_2(x)-P_1(x))}}$$ $$\qquad \qquad \qquad \qquad= \frac{1}{2}\max_{A \subset \omega}{(P_1(A)-P_2(A))} + \frac{1}{2} \max_{B \subset \omega}{(P_2(B)-P_1(B))}$$
$$\qquad \qquad \qquad \qquad= \max_{A \subset \omega}{ \mid P_1(A)-P_2(A)\mid}$$