1
$\begingroup$

Please forgive me if this is not the right Stack Exchange (I also posted it at Cross Validated and Theoretical Computer Science). Please also forgive me for inventing terms.

For discrete random variables X and Y, the mutual information of X and Y can be defined as follows: $I(X;Y) = \sum_{y \in Y} \sum_{x \in X} p(x,y) \log{ \left( \frac{p(x,y)}{p_1(x)\,p_2(y)} \right) }, \,\!$

I will define the mutual information of a "cell" $x_0$ to be: $CI(x_0,Y) = \sum_{y \in Y} p(x_0,y) \log{ \left( \frac{p(x_0,y)}{p_1(x_0)\,p_2(y)} \right) }, \,\!$

I'm not sure if this quantity goes by another name. Essentially I'm restricting focus to a single state of variable X (and then the full MI can be calculated by summing all the cell MIs).

My question: is it guaranteed that $CI(x_0,Y) \ge 0$? We know $I(X;Y)\ge0$ and we know that the pointwise mutual information can be negative. I feel like CI should be nonnegative and that I might be missing some obvious proof.

$\endgroup$
  • $\begingroup$ You should find more information in the book: "Elements of Information Theory" by Thomas and Cover. I believe there is a chapter on inequalities like these. $\endgroup$ – Aryabhata Dec 22 '10 at 18:06
1
$\begingroup$

Answered on Theoretical Computer Science. CI cannot be negative.

$\endgroup$
  • $\begingroup$ You work for Quantum Leap right? You are looking at $I/H$? $\endgroup$ – PrimeNumber Dec 22 '10 at 17:30
  • $\begingroup$ @Trevor Yes, and I'd love to know how you knew that. That's probably beyond the scope of this site though so please feel free to email me (email in my user profile). $\endgroup$ – Michael McGowan Dec 22 '10 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.