I was reading a book about techniques to pass a multiple choice test and I found a passage that seems strange to me.

Every question has 5 possible answers, you get 1 point for each correct answer, 0 for each not given answer and -.25 for each wrong answer.

The book reasoning was like this:

if you choose your answers randomly you have a probability of 1/5 to get the correct answer, so if you randomly answer five question, one will be correct.

I feel something is wrong about this deduction. I mean, since the each question will be different, how could you be sure that randomly answering five questions will get you one correct answer?

I think the correct deduction would be something like this:

if you choose your answers randomly you have a probability of 1/5 to get the correct answer, so if you randomly answer the same question for five times, you will get the correct answer with one of your answers.

The question is: is the deduction contained in the original passage from the book correct from a probabilistic standpoint?

  • $\begingroup$ yes, it's probabilistic. It's the expected number of correct answers, not the guaranteed number of correct answers. $\endgroup$ – Dustan Levenstein Jan 24 '14 at 14:43
  • $\begingroup$ I agree that the wording is poor -- it should say something like "on average you would answer one question correctly," or use the correct mathematical terminology: "The expected value of the number of correct answers is 1." $\endgroup$ – Jeff Snider Jan 24 '14 at 14:51

The book's reasoning is incorrect but the scoring policy makes good sense when interpreted properly.

On an exam, one can assume that a student who knows the correct answer to a question will mark the correct answer. If the student does not know the correct answer and marks one of the five answers at random, the student has probability $\frac15$ of marking the correct answer. Suppose that the exam has $N$ questions. The examiner knows that $C$ questions were answered correctly and $N-C$ questions were answered incorrectly. Now, the examiner is interested in assessing how many answers $K$ the student knew but the only evidence available is that $C$ out of $N$ questions were answered correctly. Of course $C \geq K$. To estimate $K$, consider that $N-C$ is the observed value of a binomial random variable with parameters $(N-K,0.8)$ since the student was guessing on $N-K$ answers. Note that the value of $N$ is known but the value of $N-K$ is not known. Now, the maximum-likelihood estimate of $K$ works out to be

$$\hat{K} = \lfloor N - 1.25 (N-C) + 1\rfloor = \lfloor C + (-0.25)(N-C) + 1\rfloor \tag{1}$$ and so if $1$ point is awarded for each of the $C$ correct answers and $-0.25$ for each of the $N-C$ wrong answers, one comes up with a good estimate of $K$, the number of answers that were marked correctly because the student knew the correct answers. Many scoring systems use $$\tilde{K} = C - \lfloor 0.25(N-C)\rfloor \tag{2}$$ as the estimate of $K$. When $N-C$ is not a multiple of 4, $\hat{K} = \tilde{K}$ and so the scoring system estimate gives the "right" answer. When $N-C$ is a multiple of 4, then both $\hat{K}$ and $\tilde{K} = \hat{K}-1$ are maximum-likelihood estimates, and use of $(2)$ just means that the student encountered a tough grader.

As an example, consider an exam with 100 questions of which the student knows the correct answers to 90. The number of wrong answers can be any number from 0 (a very lucky student who is not penalized at all!) to 10 (a very unlucky student!). On average, 92 questions are answered correctly. (This is also the most probable number of correct answers). Here $\hat{K} = 91$ while $\tilde{K} = 90$ and so the scoring system estimates the value of $K$ correctly. If the student is lucky and marks 96 correct answers, both 95 and 96 are maximum-likelihood estimates but the scoring system awards 95 points which is a better estimate of $K$ than $96$. On the other hand, the poor unlucky student who manages to mark all 10 questions incorrectly then has to suffer the further indignity of being awarded only $90 - \lfloor 0.25\times 10\rfloor = 88$ points even though he knew the correct answers to 90 questions!

  • $\begingroup$ Can you link / give a quick derivation to MLE for $K$? $\endgroup$ – user103828 Sep 27 '14 at 7:48
  • 2
    $\begingroup$ @user103828 See Problem 6 in this Problem Set for a statement of the problem. The solution (most of which I gave above) can be found here. (There is no plagiarism here; I wrote the documents cited). $\endgroup$ – Dilip Sarwate Sep 27 '14 at 12:13

No, it's technically incorrect as you can't say "one WILL be correct." It should read something like "you are expected to get one fifth of the questions right"

However, if I may deduce where the author is going with this, he will probably say something of the sort "So, you will get one question right (+1) and four incorrect answers (-1), in total you get 0 points. This way, it doesn't matter if you completely guess or skip a question. etc etc". If the book is following this reasoning, the conclusion is correct, even though the poor wording.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.