Entropy Problem: mutual information

I have a problem about entropy and mutual information that I have attempted, but would like feedback on.

1. 30% Boas
2. 20% Anaconda
3. 50% Cobra

4. Half of the Cobras were medium sized, and the other half were large.

5. One third of the Boas were small, one third of them were medium sized, the final third were large.
6. All of the Anacondas were medium sized.

c. What is her initial uncertainty about the size of the snake wrapping around her? What is the mutual information between the type of the snake and its size?

$.11 \log{1/.11} + .61 \log{1/.61} + .27 \log{1/.27}$ $= .898 nats$

Is this correct for the uncertainty on size? Three possibilities: small, med, and large, and the probability of getting each put into the $p * \log{1/p}$ formula.

I am unsure of how to solve the next part about mutual information. I think it has something to do with $I(X:Y) = H(X) - H(X|Y)$ But I don't know how to obtain H(X|Y). I could Consider H(X) My system of sizes and say H(X) = .898 correct?

migrated from physics.stackexchange.comFeb 22 '13 at 4:15

This question came from our site for active researchers, academics and students of physics.

• I think this probably belongs on math.SE - there isn't any physics in this question. – Nathaniel Feb 21 '13 at 8:35

You can find expressions for the conditional entropy on Wiki. You will need the conditional probabilities, which can be read off from your data. For example, the probability of having a medium snake, given that the snake is a cobra, is written $p(\mathrm{medium}|\mathrm{cobra}) = 1/2$. You might also need the joint probabilities, given by $p(x,y) = p(x|y)p(y)$. For example, the probability of getting a medium cobra is $p(\mathrm{medium}|\mathrm{cobra})p(\mathrm{cobra}) = 1/4$.