Markov chains and biased coin An unfair coin turns up heads with probability 0.2. It is tossed repeatedly. What is the expected number of tosses until there are two heads in a row?
How can one use Markov chains to solve this problem?
 A: Since we know the probability of this coin turning up heads is 0.2, the odds of getting two heads in a row at any point in time would be 0.2 * 0.2 = 0.04.
Now let's ask the question: How many times would we need to flip the coin to reach an expected value of 1 for this outcome? Well if someone has a probability of 0.04, we know it would have an expected outcome of 1 time after 25 trials.
Therefore the expected number of tosses to get 2 heads in a row with this unfair coin would be 25 tosses.
Now for using Markov Chains, we would have two states. An H state and a T state for heads and tails respectively. Each state would have a recursive path on themselves, since you can of course flip the same result you just flipped, and you would have a path that goes to the other option because you could also not flip the same result.
On the H state, we would have the recursive path pointing back towards itself have a value of 0.2, because the odds of flipping another heads with this unfair coin is always 0.2, regardless of the previous outcome because flips are independent. The path going from H to T has a value of 0.8. On the H state, the recursive path would also have a value of 0.8, because that's always the probability of flipping a T with this coin, and lastly the path from T to H would have a value of 0.2.
So to visualize the Markov Chain diagram, it would look like this:
H -> H : 0.2
H -> T : 0.8
T -> T : 0.8
T -> H : 0.2
