Right now I'm confused on how Information is even different than entropy.
But let's say we got an information about the result of a coin toss. How much information would that give us? In bits?
I straight up went to calculating Entropy and got 1 bit, which is easily backed up by logic, since 1 bit tells us that a value is either 1 or 0(head or tails).
How about information of a die toss result?
I'm guessing it should carry a lot more information. Using the same process as before i got 6* (1/6 * log2(1/6)) which leaves me with 2.58 bits of information. This could be right but I'm confused. Am I not just calculating regular entropy?
If by some miracle my thinking is correct for a die toss result, how much information would the result of a two-die toss result bring us? My logic would be since each die is bringing us 2.58bits of information, the sum of these should tell us how much information in an two-die-toss. What do you think?
Let's take it a bit further. If we're just randomly guessing 5 numbers out of 50, how much information would the result of that one bring us?