Estimate expected payoff of rolling a dice, with choice of rolling up to $50$ times.

This is an extended question of the classical rolling dice and give face value question.

You roll a dice, and you'll be paid by face value. If you're not satisfied, you can roll again. You are allowed $$k$$ rolls.

In the old question, if you are allowed two rolls, then the expected payoff is $$E[\text{payoff}] = 4.25$$.

If you are allowed $$3$$ rolls, the expected payoff is $$E[\text{payoff}] = 4.67$$.

If you can roll up to $$50$$ times, you can calculate the payoff using the formula and get $$E = 5.999762$$, notice that after $$5^\text{th}$$ roll, your expected payoff will be greater than $$5$$, so you'll only stop once you roll $$6$$.

So my question here is, without exact calculation(using geometric process), how would you estimate how many $$9$$s are there in the answer? Or another way to ask will be, is the expected payoff bigger than $$5.9$$? bigger than $$5.99$$? etc.

• So I get $6$ unless I roll $50$ straight times without rolling a $6$? Apr 12, 2020 at 22:25
• @saulspatz After rolling non-$6$s for the first $45$ rolls, I think the optimal player will start accepting a $5$ or a $6$, instead of just $6$s. Then, if there are only $2$ rolls left, you will accept a $4$, $5$, or a $6$. See my answer for why I think this is the case Apr 12, 2020 at 23:13
• @ZubinMukerjee Thanks. I was asking actually asking what the rules are, but I guess there's really only one interpretation that makes sense. Apr 13, 2020 at 0:13

Let $$E_k$$ be the expected payoff, if you're allowed to roll $$k$$ times, with the rules as you've described them. We can compute $$E_k$$ recursively.

With just $$1$$ roll, you must take what you get, since there are no more rolls. The expected value is therefore $$E_1 = \frac{1+2+3+4+5+6}{6} = 3.5$$

With $$2$$ rolls, if your first roll is $$4$$, $$5$$, or $$6$$, you will keep it, otherwise you will reroll and get $$E_1$$ from your next (and last) roll. Therefore, \begin{align*}E_2 &= \frac{4+5+6}{6}+\frac{1}{2}E_1 \\ &= 2.5+\frac{1}{2}(3.5) = 4.25\end{align*}

With $$3$$ rolls, if your first roll is $$5$$ or $$6$$, then you will keep it, otherwise you will reroll and get $$E_2$$ from your next two rolls. Therefore, \begin{align*} E_3 &= \frac{5+6}{6}+\frac{2}{3}E_2\\ &= \frac{11}{6}+\frac{2}{3}(4.25) = 4.\overline{6} \end{align*}

With $$4$$ rolls, if your first roll is $$5$$ or $$6$$, then you will keep it, otherwise you will reroll and get $$E_3$$ from your next three rolls. Therefore, \begin{align*} E_4 &= \frac{5+6}{6}+\frac{2}{3}E_3\\ &= \frac{11}{6}+\frac{2}{3}(4.\overline{6}) = 4.9\overline{4} \end{align*}

With $$5$$ rolls, if your first roll is $$5$$ or $$6$$, then you will keep it, otherwise you will reroll and get $$E_4$$ from your next three rolls. Therefore, \begin{align*} E_5 &= \frac{5+6}{6}+\frac{2}{3}E_4\\ &= \frac{11}{6}+\frac{2}{3}(4.9\overline{4}) = 5.1\overline{296} = \frac{277}{54} \end{align*}

Now, we have reached the point at which the recursion relation is stable. With more than $$5$$ rolls, you will always only keep the first roll if it is a $$6$$.

With $$k$$ rolls, $$k>5$$ if your first roll is $$6$$, you will keep it, otherwise you will reroll and get $$E_{k-1}$$ from the next $$k-1$$ rolls. Therefore,\begin{align*} E_k &= \frac{6}{6}+\frac{5}{6}E_{k-1}\\ E_k &= 1+\frac{5}{6}E_{k-1}\tag{1}\\\ \end{align*}

Notice that $$E_5 = \frac{277}{54} = 6 - \frac{47}{54}$$

The solution to the recurrence relation in $$(1)$$, with initial value $$E_5 = 6- 47/54$$, is:

$$E_k = 6 - \left(\frac{47 \cdot 144}{5^5}\left(\frac{5}{6}\right)^k\right)$$

Therefore, in general, the maximum expected payoff that you can achieve, when allowed $$k$$ rolls of a six-sided die, for any $$k$$, is $$\boxed{\,\,E_k \,=\,\begin{cases}7/2 \qquad &\text{if}\,\,\,k=1\phantom{l^{l^{l^{\overline{l}}}}}\\ 17/4 \qquad &\text{if}\,\,\,k=2\phantom{l^{l^{l^{\overline{l}}}}}\\ 14/3 \qquad &\text{if}\,\,\,k=3\phantom{l^{l^{l^{\overline{l}}}}}\\ 89/18 \qquad &\text{if}\,\,\,k=4\phantom{l^{l^{l^{\overline{l}}}}}\\\\6-\displaystyle\frac{6768}{3125}\left(\displaystyle\frac{5}{6}\right)^k \qquad &\text{if}\,\,\,k\geq 5\phantom{l_{l_{l_{l_l}}}}\\ \end{cases}\,\,\,}$$

Let $$a_n$$ be the expected payoff of an $$n$$-roll game. We have $$a_1=3.5$$ and the recursion $$a_{n+1} = \frac{6 + \lceil a_n \rceil}{2} \cdot \frac{7 - \lceil a_n \rceil}{6} + a_n \cdot \frac{\lceil a_n \rceil - 1}{6}$$

You noted that for $$n \ge 5$$ we have $$\lceil a_n \rceil = 6$$, so the recursion in that case becomes $$a_{n+1} = 1 + a_n \cdot \frac{5}{6},\qquad n \ge 5.$$

Letting $$p = 5/6$$ we have we have the general formula \begin{align} a_n &= p^{n-5} a_5 + p^{n-6} + p^{n-7} + \cdots + p + 1 \\ &= p^{n-5} a_5 + \frac{1-p^{n-5}}{1-p} \\ &= (5/6)^{n-5} a_5 + 6(1-(5/6)^{n-5}) \\ &= 6 - (5/6)^{n-5} (6 - a_5) \end{align} for $$n \ge 5$$.

The second term $$(5/6)^{n-5} (6 - a_5)$$ tells you how far the expected payoff is from $$6$$; you can set this to $$0.1$$ or $$0.01$$ and solve for $$n$$ to answer your question.