Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How to prove the following known (Pinsker's) inequality?

For two strictly positive sequences $(p_i)^n_{i=l}$ and $(q_i)^n_{i=l}$ with $\sum_{i=1}^np_i=\sum_{i=1}^nq_i=1$ one has $$\sum_{i=1}^np_i\log\frac{p_i}{q_i}\ge \frac{1}{2}\left(\sum_{i=1}^n|p_i-q_i|\right)^2.$$

share|cite|improve this question
Retagged due to the relation to Kullback-Leibler_divergencecomments may only be edited for 5 minutes(click on this box to dismiss) – Ilya Apr 4 '12 at 13:14
If it is known, does it have a name? Where did you find it and was a reference to a proof missing? – Aryabhata Apr 4 '12 at 17:15
@Aryabhata yes it is known. Pinsker's inequality:… – Kolmo Apr 4 '12 at 18:35
@Kolmo: Please add an answer. Also, my point was OP knows it is known, and if OP knew the name, they could have done some research before posting it here. – Aryabhata Apr 4 '12 at 18:51
Check also Beck & Teboulle 2003, "Mirror descent and nonlinear projected subgradient methods for convex optimization", Proposition 5.1 for elementary proof of a weaker inequality with symmetrized KL-divergence. – ostrodmit Sep 15 '15 at 9:51
up vote 3 down vote accepted

See here. Pinsker's inequality

share|cite|improve this answer
Thanks...... +1 – Aryabhata Apr 4 '12 at 19:35
Is there a simpler proof? – Sunni Apr 4 '12 at 20:17
I find a simpler one from Borwein\& Levis, 2005, page 63. – Sunni Apr 10 '12 at 19:25

Take a look at Tsybakov 2009 Introduction to Nonparametric Estimation, p.88, also available in the net. Good and simple proof.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.