1
$\begingroup$

Consider a multiple-choice (a, b, c, d) test with 10 questions

  • If you choose answers randomly, the expected grade is $25\%$
  • If you fill out two tests randomly and pick the best grade, the expected grade is $33\%$.
  • If you take the best among 10 random tests, the expected grade is $\approx47\%$.
  • If you take the best among 100 random tests, the expected grade is $\approx62\%$.
  • If you take the best among 1000 random tests, the expected grade is $\approx72\%$.
  • If you take the best among 10000 random tests, the expected grade is $\approx82\%$.
  • But on new questions, the random choice accuracy is still $25\%$.

I am confused about how $33\%$ and the rest of the expected grades are calculated. It is obvious the problem assumes questions are iid. For a single test, it is a Bernoulli trial. the number of questions isn't a factor here, so $E=\frac{1}{4}$.

It is unclear the setup of the experiment, I assume we are in a $\binom{N}{2}$ scenario, where $N$ is the number of all combinations, in this case, $4^{10}=1048576$, which mean the random answers cannot be the same. However, this does not explain how $33\%$ is computed.

This problem came from the context of optimization bias, but no formula was given.

  • Is the problem missing some conditions? If so, what are the missing conditions, and how to make the problem more concrete?
  • Which distribution does this question fit into?
  • How these expected values are computed?
$\endgroup$
4
  • 2
    $\begingroup$ The calculation of the expectation can be found with powers of the cumulative distribution function. For example in R, you can use sum((1-pbinom(0:10,10,1/4)^2))/10 to get 0.3256158 rounding to $33\%$ and changing the 2 to however many tests there are will give you the other results $\endgroup$
    – Henry
    Mar 11, 2022 at 14:07
  • $\begingroup$ @Henry Could you explain a little bit of where this equation comes from? And why is it /10 instead of /11, since there are 11 possible outcomes 0:10? $\endgroup$
    – FisNaN
    Mar 12, 2022 at 10:21
  • $\begingroup$ If you do not divide by $10$ then you get the expected number of correct answers in the best attempt, $3.256158$ in the two test case with $10$ questions, so dividing by the number of questions turns this into a proportion. $\endgroup$
    – Henry
    Mar 12, 2022 at 10:33
  • 1
    $\begingroup$ The rest of the code uses a complementary CDF-related method of calculating the mean of a non-negative integer random variable $\mathbb E[X] = \sum_n \mathbb P(X>n)$ and combining this with the fact that the probability none of the tests exceed a given mark is the the power of the probability a given test does not exceed it $\endgroup$
    – Henry
    Mar 12, 2022 at 11:02

1 Answer 1

1
$\begingroup$

Each question is a Bernoulli random variable. Each test is a Binomial random variable. We are looking at the maximum of a number of tests. That is, if $B_i\sim\text{Binomial}(10, 0.25)$ are i.i.d. then the values given are $\mathbb{E}(\max\{B_1,B_2,\dots,B_m\})$ for different values of $m$ ($1,2,10,100,1000,10000$).

The only assumptions made are that we are filling out every question on every test independently and uniformly at random (each option has a $25$% chance of being picked).

The random variable $\max\{B_1,B_2,\dots,B_n\}$ is very messy: see here for the distribution function (in the slightly more general case that the success probability $p$ can vary).

As we have been given $n=10,p=0.25$ and values of $m$ explicitly, you can do explicit calculation to find exact values of the expectation. Or, you might just run each experiment some large number of times (on a computer) and take the mean. (For instance, for $m=10$, you could run $10^6$ trials where you take $10$ i.i.d. $\text{Binomial}(10,0.25)$ r.v.s and find the maximum of those $10$, and compute the mean across all trials.)

To give an example of a calculation: in the case $m=10$, we see that the probability of scoring $\le 3$ is the probability that each of the ten tests have a score $\le 3$. One test has a score of $\le 3$ w.p. $0.25^{10}+10\times 0.75\times 0.25^9+{10\choose 2}\times 0.75^2\times 0.25^8+{10\choose 3}\times 0.75^3\times 0.25^7$: call this number $p$. Then the probability that this occurs $10$ times without fail is $p^{10}$. We could do similar calculations to get the probability of scoring $\le 0, \le 1, \dots, \le 9$ and then $\mathbb{P}(\text{we score}\ i\ \text{exactly})=\mathbb{P}(\text{we score}\le i)-\mathbb{P}(\text{we score}\le i-1)$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .