Imagine there's a guessing game where a series of binary symbols are presented and a human must decide quickly if the symbol is the same as the previous or different. There's a property of the sequence which I'm trying to define mathematically which corresponds to how difficult it is to guess whether the next symbol will be the same or different. It's partly entropy, but..
On the one hand, the sequence 010101... has maximal entropy per bit. On the other hand, it is an unchanging series of the word '01' - not very confounding. (I use the word 'confounding' because I'm not sure if it's correct to say its 'second order entropy would be zero'). How can you maximize entropy across all possible word lengths at once ? What's an optimal guessing algorithm that works well assuming that the game creator wants to use the most 'confounding' sequence ? What if you can't know the confounding-ness of the sequence up-front ?
I'm reading up on Huffman encodings but any pointers to other similar topics would be great. I'm trying to find out what exactly is this notion of 'confounding' that I'm describing in real mathematical terms ?