I'm trying to figure out what this question is asking and what it is I'm trying to calculate exactly.

I'm told: You have cards 2-5 of each suit, except the 2 and 3 of the red cards. So 12 cards total.

I've calculated: All the probability distributions (conditional, joint), the entropies (conditional, joint), mutual information.

It sounds like the question is NOT asking for a conditional entropy? Or am I understanding entropy incorrectly? It's an AVERAGE amount of information, right? I'm not too sure what I'm trying to calculate...

  • $\begingroup$ Did you mean "of each suit" and "of the red cards"? $\endgroup$ – Christian Blatter Sep 16 '13 at 19:23
  • $\begingroup$ Yes I did. Thanks. I fixed it. $\endgroup$ – user1399747 Sep 17 '13 at 1:15

Learning that the card is red takes you from 14 (not 12) equally likely possibilities to two equally likely possibilities. So now you are one bit short of complete knowledge of the card, whereas you originally were $\log_1 14$ bits short. You won $\log_214-1$ bits of information.

  • $\begingroup$ Thank you. I think I get it. I understand that there is 1 bit of uncertainty left. The log 14 is the amount of bits needed to represent one card choice (out of 14 cards)? If I had picked a black card instead of a red one, I would have only gained 1 bit of info and been left with an uncertainty of (log 14 - 1)? $\endgroup$ – user1399747 Sep 17 '13 at 1:22
  • $\begingroup$ @Hagen von Eitzen: At the beginning there are $4\times 4-2\times2=12$ cards. $\endgroup$ – Christian Blatter Sep 17 '13 at 6:32
  • $\begingroup$ Ok, so then it would be log 12 total uncertainty, and if you pick one card that's red, you've eliminated 4 cards, so log 4 = 2 bits? leaving you with log 12 - 2 ? But wait, that doesn't make sense because you can't do the reverse, and log 4 + log 8 != log 12. $\endgroup$ – user1399747 Sep 17 '13 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.