# Proof of Pinsker's inequality.

How to prove the following known (Pinsker's) inequality?

For two strictly positive sequences $(p_i)^n_{i=l}$ and $(q_i)^n_{i=l}$ with $\sum_{i=1}^np_i=\sum_{i=1}^nq_i=1$ one has $$\sum_{i=1}^np_i\log\frac{p_i}{q_i}\ge \frac{1}{2}\left(\sum_{i=1}^n|p_i-q_i|\right)^2.$$

• Retagged due to the relation to Kullback-Leibler_divergencecomments may only be edited for 5 minutes(click on this box to dismiss)
– Ilya
Apr 4, 2012 at 13:14
• If it is known, does it have a name? Where did you find it and was a reference to a proof missing? Apr 4, 2012 at 17:15
• @Aryabhata yes it is known. Pinsker's inequality: mathoverflow.net/questions/42667/… Apr 4, 2012 at 18:35
• @Kolmo: Please add an answer. Also, my point was OP knows it is known, and if OP knew the name, they could have done some research before posting it here. Apr 4, 2012 at 18:51
• Check also Beck & Teboulle 2003, "Mirror descent and nonlinear projected subgradient methods for convex optimization", Proposition 5.1 for elementary proof of a weaker inequality with symmetrized KL-divergence. Sep 15, 2015 at 9:51

## 2 Answers

See here. Pinsker's inequality

https://mathoverflow.net/questions/42667/two-reference-requests-pinskers-inequality-and-pontryagin-duality

• Is there a simpler proof? Apr 4, 2012 at 20:17
• I find a simpler one from Borwein\& Levis, 2005, page 63. Apr 10, 2012 at 19:25
• @Kolmo can you please point out the condition for equality in Pinsker's inequality? Nov 12, 2016 at 9:14

Take a look at Tsybakov 2009 Introduction to Nonparametric Estimation, p.88, also available in the net. Good and simple proof.