I came across an interesting dice roll game that involved expected values and wanted to know if my approach and answer were correct, and if there are other ways to solve this.
Problem statement:
You're playing a game, as follows. On the table is a pile of money, currently containing a single dollar. On every turn, you have two options. The first option is to simply take the money on the table, ending the game. The second option is to roll a die. If you roll a 6, all the money on the table disappears and the game ends. Otherwise, a dollar gets added to the pile on the table and the game continues.
What's the optimal strategy for this game?
Let $f(x)$ represent the expected payoff from this game after $x$ rolls of the dice. Using this, I can calculate a few values:
If we choose not to roll the dice at all, the payoff is just $\$1$. Thus, $f(0) = 1$.
If we choose to roll the dice once, there is a $\frac{5}{6}$ chance our payoff is $\$2$ and a $\frac{1}{6}$ chance our payoff is $\$0$ if we roll a 6. As a result, $f(1) = \frac{10}{6}$.
If we choose to roll the dice twice, there is a $\frac{25}{36}$ chance that we get two non-sixes in which case our payoff is $\$3$, and it is $\$0$ for all other cases. Therefore, $f(2) = \frac{75}{36}$.
Based on this pattern, I can come up with a general expression:
$$f(x) = \left(\frac{5}{6}\right)^x (x+1)$$
To determine the optimal strategy, I should seek to find the number of dice rolls $x$ that maximizes $f(x)$. The obvious way to do so is to take the derivative with respect to $x$ and set it equal to $0$:
$$f'(x) = \left(\frac{5}{6}\right)^x \ln\left(\frac{5}{6}\right)(x+1) + \left(\frac{5}{6}\right)^x = 0.$$ $$\ln\left(\frac{5}{6}\right)(x+1) = -1$$ $$x = \frac{-1}{\ln\left(\frac{5}{6}\right)} - 1 \approx 4.4848.$$
The closest integers are $4$ and $5$ and on further inspection, $f(4) = f(5)$. Thus, the optimal strategy is to roll the dice either $4$ or $5$ times.
Is this a correct/valid way to approach this problem? Is there another way? Thank you.
