Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Please forgive me if this is not the right Stack Exchange (I also posted it at Cross Validated and Theoretical Computer Science). Please also forgive me for inventing terms.

For discrete random variables X and Y, the mutual information of X and Y can be defined as follows: $I(X;Y) = \sum_{y \in Y} \sum_{x \in X} p(x,y) \log{ \left( \frac{p(x,y)}{p_1(x)\,p_2(y)} \right) }, \,\!$

I will define the mutual information of a "cell" $x_0$ to be: $CI(x_0,Y) = \sum_{y \in Y} p(x_0,y) \log{ \left( \frac{p(x_0,y)}{p_1(x_0)\,p_2(y)} \right) }, \,\!$

I'm not sure if this quantity goes by another name. Essentially I'm restricting focus to a single state of variable X (and then the full MI can be calculated by summing all the cell MIs).

My question: is it guaranteed that $CI(x_0,Y) \ge 0$? We know $I(X;Y)\ge0$ and we know that the pointwise mutual information can be negative. I feel like CI should be nonnegative and that I might be missing some obvious proof.

share|cite|improve this question
You should find more information in the book: "Elements of Information Theory" by Thomas and Cover. I believe there is a chapter on inequalities like these. – Aryabhata Dec 22 '10 at 18:06
up vote 1 down vote accepted

Answered on Theoretical Computer Science. CI cannot be negative.

share|cite|improve this answer
You work for Quantum Leap right? You are looking at $I/H$? – PEV Dec 22 '10 at 17:30
@Trevor Yes, and I'd love to know how you knew that. That's probably beyond the scope of this site though so please feel free to email me (email in my user profile). – Michael McGowan Dec 22 '10 at 17:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.