# Dice Game: Optimal Strategy

Recently, we played a simple dice game at school. The objective of the game is to gain as many points as possible (have the largest expected value of points). Each round consists of several turns. At each turn, the player rolls a dice. Then,

1. If the die is not a two, the player adds the number to his pot.
2. If the die is a two, the player loses everything and the round ends.
3. At the end of each roll, the player is given a chance to cash out, and receive the pot.

Finally, the player gets a free roll at the beginning (if the roll is a two, it is still added to the pot). What is the optimal strategy for winning?

I thought of two different ways to answer this question, but they gave different results.

1) Let $$s_{t}$$ be the current size of the pot. The expected value of taking the next turn is $$\mathbb{E}[s_{t+1}] = 5/6 * (s_{t} + 19 / 5)$$ where 19/5 is the expected value given that the roll is not a two, $$19 / 5 = (1+3+4+5+6)/ 5$$. Therefore, the time to stop, or when the expectation as its maximum, is when $$s_t$$ equals $$\mathbb{E}[s_{t+1}]$$, which is $$s_{t}= 19$$.

2) Instead of looking at the final score, I looked at what number of turns gives the best return. The equation I created was this: $$(19 / 5 n + 21 / 6) * (5/6) ^ n$$where n is the number of moves taken after the first free roll. The first part of the equation is the expected number of points gained from n moves given that each roll was not a two. The second part gives the probability of the person surviving until the nth decision. The maximum is obtained at around n = 4.38 with the equation equal to 9.38.

Here's where I try to compare the method 1 to method 2: Since the each roll contributes an expected value of (19 / 5), the expected number of decisions to make to reach 20 is $$19 / (19/5) - 1$$ or 4. That is not equal to the number of moves obtained from the second method, 4.38.

Are any of these methods valid arguments? Is the comparison valid? If not, are the two methods equivalent to each other?

The first argument is answering, "If I am facing a single decision on a single turn with $$s_t$$ in the pot, do I expect to make more by cashing out now or by cashing in after 1 more round?" dependent on the size of the pot.
The second argument is answering the question, "If I decide in advance that I'm going to play $$n$$ turns and then cash out, how much do I expect to make?" and then picking the $$n$$ that maximizes this. This throws away your ability to reevaluate each turn depending on new information. Staying in with $18 in the pot is a good idea, whether it's turn 1 or turn 7. But be careful, games like this get a bit weird if they go on for a while. For example, suppose you start with a dollar and flip a coin, heads you triple your current total and tails you lose it all. Expected value for the first round is +$0.50, so you should play. And then if you win, why not play again? And again? But if you keep playing indefinitely, you're almost sure to lose everything. How many rounds do you play? See, for comparison, the St. Petersburg Paradox