You are given a biased coin with probability $p$ of getting $H$ and $1-p$ of getting tail. Each flip is independent of another. We keep flipping the coin until we get $4$ consecutive tails. For each tail, we get a $2$ dollars and for each head we lose $1$ dollar. What is the $p$ that would maximize the amount of money we have?
$\begingroup$
$\endgroup$
-
1$\begingroup$ @AakashM Yes, $p>0$ delays the end of the game with the opportunity to win more (and hopefully not to lose too much). $\endgroup$ – Hagen von Eitzen Jan 18 '13 at 11:52
-
$\begingroup$ @HagenvonEitzen the early history of probability theory should have warned me not to trust intuition about gambling :) $\endgroup$ – AakashM Jan 18 '13 at 11:57
-
$\begingroup$ @AakashM You should especially never trust the institution (i.e. the bank holder) about gambling :) $\endgroup$ – Hagen von Eitzen Jan 18 '13 at 12:27
$\begingroup$
$\endgroup$
Let $E_k$ be the expected win if we start with a history of $k$ tails. We want to maximize $E_0$.
We have $$E_3 = p\cdot(-1+E_0) + (1-p)\cdot 2$$ as we either loose a dollar and start from $k=0$ again, or win two dollars and stop. Similarly $$E_2 = p\cdot(-1+E_0) + (1-p)\cdot (2+E_3)$$ $$E_1 = p\cdot(-1+E_0) + (1-p)\cdot (2+E_2)$$ $$E_0 = p\cdot(-1+E_0) + (1-p)\cdot (2+E_1).$$ Combining these equations, we find $$E_0= \frac{3p^4-14p^3+26p^2-24p+8}{(p-1)^4}.$$ The derivative of this is $$E_0'(p) =\frac{p^3-5p^2+10p-4}{(p-1)^5} $$ and has a single root in the interval $[0,1]$ at $p\approx 0.522$, which corresponds to a local maximum value of $\approx15.09$.
answered Jan 18 '13 at 11:49
