I am trying to understand the problem of expected payoff of a dice game explained here. I can roll the dice up to three times, but after each one I decide if I want to try once again or not. The idea is to find an optimal strategy that maximizes the expected payoff (expected number of spots; they do not sum up).

Let's say I know the optimal strategy: If in the first roll I get 1,2 or 3 I roll the dice once again. If in the second I get 1,2,3 or 4 I roll the dice once again. The end.

I wanted to calculate the expected number using probability $P(X=k)$ that I end up with $k$ spots. I am probably doing something wrong because I don't get the correct answer. My reasoning is the following:

$$P(X=\{1,2,3\}) = \underbrace{\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}}_{\text{3rd roll}}$$ $$P(X=4) = \underbrace{\frac{1}{6}}_{\text{1st roll}}+\underbrace{\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}}_{\text{3rd roll}}$$
$$P(X=5) = \underbrace{\frac{1}{6}}_{\text{1st roll}}+\underbrace{\frac{3}{6}\cdot\frac{1}{6}}_{\text{2nd roll}}+\underbrace{\frac{3}{6} \cdot\frac{4}{6}\cdot\frac{1}{6}}_{\text{3rd roll}}$$ $$P(X=6) = \frac{1}{6} + \frac{3}{6}\cdot\frac{1}{6}+\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}$$

Expected number is $E[X] = \sum_{k=1}^{6}kP(X=k) \approx 4.58$, which is not correct.

I appreciate any help.

  • 1
    $\begingroup$ Are you sure you don’t have that strategy backwards? You should reroll on 4 or less for the first roll, and 3 or less for the second. When you have fewer rolls you have left, you should be less willing to reroll. $\endgroup$ – Mike Earnest May 19 '18 at 15:16
  • $\begingroup$ @MikeEarnest In order to end the game with e.g. $1$. You should roll 1,2 or 3 in the first try ($p=3/6$). Then you should roll 1,2,3, or 4 in the second ($p=4/6$). You get $1$ with probability $p=1/6$ in the last roll. $\endgroup$ – WoofDoggy May 19 '18 at 15:24
  • $\begingroup$ Your calculations are correct in this post, I am not disputing that. However, you are basing your calculations on an optimal strategy that someone told you, and I think that that strategy you heard is incorrect, which could perhaps explain the error. $\endgroup$ – Mike Earnest May 19 '18 at 15:40
  • $\begingroup$ I wrote a simple program in C++ in order to check if probabilities are correct. Yes, they are. Now I am puzzled. The recurrence formula given by @Ben gives $4.6333$ instead of $4.58$. Moreover, in my approach I do not see a recurrence. $\endgroup$ – WoofDoggy May 19 '18 at 16:44
  • $\begingroup$ Ok, I start to understand what is wrong with my reasoning after reading this. What I calculated is correct, but as mentioned by @MikeEarnest the optimal strategy is different. My strategy is fixed and that is the problem. $\endgroup$ – WoofDoggy May 19 '18 at 16:56

Using your method:

$1$-roll game: $$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{1}{6}+2\cdot \frac16+3\cdot \frac16+4\cdot \frac16+5\cdot \frac16+6\cdot \frac16=3.5.$$ $2$-roll game:
$$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{3}{36}+2\cdot \frac{3}{36}+3\cdot \frac{3}{36}+\\ 4\cdot \left(\frac16+\frac{3}{36}\right)+5\cdot \left(\frac16+\frac{3}{36}\right)+6\cdot \left(\frac16+\frac{3}{36}\right)=4.25.$$ Note: The player rerolls if the first roll was $1,2,3$.

$3$-roll game: $$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{12}{216}+2\cdot \frac{12}{216}+3\cdot \frac{12}{216}+\\ 4\cdot \left(\frac{4}{36}+\frac{12}{216}\right)+5\cdot \left(\frac16+\frac{4}{36}+\frac{12}{216}\right)+6\cdot \left(\frac16+\frac{4}{36}+\frac{12}{216}\right)=4\frac23.$$ Note: The player rerolls if the first roll is $1,2,3,4$ and the second roll is $1,2,3$. You did the other way around!


I think you can approach this type of question inductively. Suppose you use an optimal stragedy for n rolls the expected value is $e_n$. Then for $n+1$ rolls, the expected value $e_{n+1}=\sum_{i: i\le e_n} \frac{1}{6} e_n+ \sum_{i: i>e_n} \frac{1}{6}(i)$. If your first roll is greater than $e_n$ then you dont roll anymore because you already get a value bigger than the expected optimal value for n rolls and if the value of first roll is smaller than $e_n$, then you are in the situation of n rolls and you keep performing the stragedy because the expected value for n rolls is higher than the firet roll.

  • $\begingroup$ Thanks @Ben. My trouble is that I do not see the recurrence. For this reason I wanted to try the basic approach using probability. Will the recurrence hold if I do not chose the optimal strategy? For example I always roll the dice if I get 1,2,3,4 or 5 (the formula will change of course). $\endgroup$ – WoofDoggy May 19 '18 at 15:30
  • $\begingroup$ You can prove inductively that the stragedy I wrote down is optimal. The recurrence relation is determined by the stragedy so different stragedy will give you different relation. $\endgroup$ – Ben May 19 '18 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.