Expected Value question regarding rolling a die multiple times

Question

Let's say we play a game where we can roll a die up to two times. After each roll you may take the value you rolled, or roll again. What is the expected value assuming perfect play?

Intuitively (not sure how to prove this is the 'perfect play') I get this:

$\frac{1}{2} * \frac{4 + 5 + 6}{3} + \frac{1}{2} * \frac{1 + 2 + 3 + 4 + 5 + 6}{6} = 4.25$

Basically, if you don't get a 4, 5, or 6 on your first roll you would roll again.

Let's extend the game to a maximum of 3x rolls:

Then you would re-roll your first roll if you don't get a 5 or 6, and the rest of the result breaks down into the 2 die game. Again, I'm not sure how to prove this is 'perfect play'.

$\frac{1}{3} * \frac{5 + 6}{2} + \frac{2}{3} * 4.25 = 4.\overline{66}$

My question is then two-part:

1.) How do we prove the above play is optimal?

2.) What happens in the case of 4+ die? For cases 2 and 3 it is easy since 1/2 and 1/3 probability easily divides 6 numbers. I'm not sure how to extend this for the case of 4 die (and other numbers of die that don't easily divide 6).

Answer 1

You've got everything right but just to state things a little more mathematically, the expected value of $X_1$ our score given one roll is $E[X_1]=\sum_{i=1}^{6}p(X=i)i = \frac{7}{2}$. Therefore given two rolls, if we get more than $\frac{7}{2}$ on the first we take the first roll, if not we roll again and can expect to get $\frac{7}{2}$. Therefore the expected value of $X_2$ our score given two rolls is $E[X_2]=p(x>\frac{7}{2})\frac{4+5+6}{3}+p(x\leq\frac{7}{2})\frac{7}{2}=\frac{17}{4}$. And for three rolls we only take the first if it is greater than $\frac{17}{4}$ as we expect to get $\frac{17}{4}$ with our remaining two rolls. The expected value of $X_3$ is therefore $E[X_3]=p(x>\frac{17}{4})\frac{5+6}{2}+p(x\leq\frac{17}{4})\frac{17}{4}=\frac{14}{3}$.

Now for four and more dice the logic is the exact same: we only take the first roll if the value is greater than the expected value of the rolls that come afterwards.

So for example $E[X_4]=p(x>\frac{14}{3})\frac{5+6}{2}+p(x\leq\frac{14}{3})\frac{14}{3}=\frac{89}{18}$.