1
$\begingroup$

I am trying to understand the problem of expected payoff of a dice game explained here. I can roll the dice up to three times, but after each one I decide if I want to try once again or not. The idea is to find an optimal strategy that maximizes the expected payoff (expected number of spots; they do not sum up).

Let's say I know the optimal strategy: If in the first roll I get 1,2 or 3 I roll the dice once again. If in the second I get 1,2,3 or 4 I roll the dice once again. The end.

I wanted to calculate the expected number using probability $P(X=k)$ that I end up with $k$ spots. I am probably doing something wrong because I don't get the correct answer. My reasoning is the following:

$$P(X=\{1,2,3\}) = \underbrace{\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}}_{\text{3rd roll}}$$ $$P(X=4) = \underbrace{\frac{1}{6}}_{\text{1st roll}}+\underbrace{\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}}_{\text{3rd roll}}$$
$$P(X=5) = \underbrace{\frac{1}{6}}_{\text{1st roll}}+\underbrace{\frac{3}{6}\cdot\frac{1}{6}}_{\text{2nd roll}}+\underbrace{\frac{3}{6} \cdot\frac{4}{6}\cdot\frac{1}{6}}_{\text{3rd roll}}$$ $$P(X=6) = \frac{1}{6} + \frac{3}{6}\cdot\frac{1}{6}+\frac{3}{6} \cdot \frac{4}{6} \cdot\frac{1}{6}$$

Expected number is $E[X] = \sum_{k=1}^{6}kP(X=k) \approx 4.58$, which is not correct.

I appreciate any help.

$\endgroup$
5
  • 1
    $\begingroup$ Are you sure you don’t have that strategy backwards? You should reroll on 4 or less for the first roll, and 3 or less for the second. When you have fewer rolls you have left, you should be less willing to reroll. $\endgroup$ Commented May 19, 2018 at 15:16
  • $\begingroup$ @MikeEarnest In order to end the game with e.g. $1$. You should roll 1,2 or 3 in the first try ($p=3/6$). Then you should roll 1,2,3, or 4 in the second ($p=4/6$). You get $1$ with probability $p=1/6$ in the last roll. $\endgroup$
    – WoofDoggy
    Commented May 19, 2018 at 15:24
  • $\begingroup$ Your calculations are correct in this post, I am not disputing that. However, you are basing your calculations on an optimal strategy that someone told you, and I think that that strategy you heard is incorrect, which could perhaps explain the error. $\endgroup$ Commented May 19, 2018 at 15:40
  • $\begingroup$ I wrote a simple program in C++ in order to check if probabilities are correct. Yes, they are. Now I am puzzled. The recurrence formula given by @Ben gives $4.6333$ instead of $4.58$. Moreover, in my approach I do not see a recurrence. $\endgroup$
    – WoofDoggy
    Commented May 19, 2018 at 16:44
  • $\begingroup$ Ok, I start to understand what is wrong with my reasoning after reading this. What I calculated is correct, but as mentioned by @MikeEarnest the optimal strategy is different. My strategy is fixed and that is the problem. $\endgroup$
    – WoofDoggy
    Commented May 19, 2018 at 16:56

2 Answers 2

3
$\begingroup$

I think you can approach this type of question inductively. Suppose you use an optimal stragedy for n rolls the expected value is $e_n$. Then for $n+1$ rolls, the expected value $e_{n+1}=\sum_{i: i\le e_n} \frac{1}{6} e_n+ \sum_{i: i>e_n} \frac{1}{6}(i)$. If your first roll is greater than $e_n$ then you dont roll anymore because you already get a value bigger than the expected optimal value for n rolls and if the value of first roll is smaller than $e_n$, then you are in the situation of n rolls and you keep performing the stragedy because the expected value for n rolls is higher than the firet roll.

$\endgroup$
2
  • $\begingroup$ Thanks @Ben. My trouble is that I do not see the recurrence. For this reason I wanted to try the basic approach using probability. Will the recurrence hold if I do not chose the optimal strategy? For example I always roll the dice if I get 1,2,3,4 or 5 (the formula will change of course). $\endgroup$
    – WoofDoggy
    Commented May 19, 2018 at 15:30
  • $\begingroup$ You can prove inductively that the stragedy I wrote down is optimal. The recurrence relation is determined by the stragedy so different stragedy will give you different relation. $\endgroup$
    – Ben
    Commented May 19, 2018 at 15:38
1
$\begingroup$

Using your method:

$1$-roll game: $$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{1}{6}+2\cdot \frac16+3\cdot \frac16+4\cdot \frac16+5\cdot \frac16+6\cdot \frac16=3.5.$$ $2$-roll game:
$$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{3}{36}+2\cdot \frac{3}{36}+3\cdot \frac{3}{36}+\\ 4\cdot \left(\frac16+\frac{3}{36}\right)+5\cdot \left(\frac16+\frac{3}{36}\right)+6\cdot \left(\frac16+\frac{3}{36}\right)=4.25.$$ Note: The player rerolls if the first roll was $1,2,3$.

$3$-roll game: $$E(X)=\sum_{k=1}^6 kP(X=k)=1\cdot \frac{12}{216}+2\cdot \frac{12}{216}+3\cdot \frac{12}{216}+\\ 4\cdot \left(\frac{4}{36}+\frac{12}{216}\right)+5\cdot \left(\frac16+\frac{4}{36}+\frac{12}{216}\right)+6\cdot \left(\frac16+\frac{4}{36}+\frac{12}{216}\right)=4\frac23.$$ Note: The player rerolls if the first roll is $1,2,3,4$ and the second roll is $1,2,3$. You did the other way around!

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .