Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose $p$ and $q$ are discrete distributions over $k$ outcomes, $q$ is given, for which $p$ does the following hold?

\begin{equation*} p_1 \log q_1 + \ldots + p_k \log q_k \ge q_1 \log q_1 + \ldots + q_k \log q_k\end{equation*}

share|cite|improve this question
up vote 4 down vote accepted

p such that:

$$\prod_i q_i^{p_i-q_i} \ge 1$$

I arrive at this point with the following calculations:

First of all I rewrite the equation:

$$\sum_i p_i \log q_i \ge \sum_i q_i \log q_i$$


$$\sum_i (p_i \log q_i - q_i \log q_i) = \sum_i (\log q_i^{p_i-q_i}) = \log \prod_i q_i^{p_i-q_i} \ge 0$$

The most simple deduction about p is that

$$p_i \ge q_i , \forall i$$


$$q_i \ge 1$$

share|cite|improve this answer
Welcome to math.SE! We support TeX equations here, but they have to be surrounded by dollar signs. See… for detail. – kennytm Aug 20 '10 at 12:34

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.