Assume I want to pass a test with 98 % probability, or 95 % probability or 90 % probability or even as low as with 75 % probability.

We say that I will take the test if the probability of success is at least 75 %.

The test has 80 questions. (It is the OCAJP 8 test.) I have a practice test with 77 questions. Now if I'm lazy I don't answer all 77 questions on the practice (mock) test but only as many as I need to estimate my probability of success.

For instance, I answer 4 mock questions and get 3 answers right, then it is too little sample size to assume that I will pass the test (where passing score is 65 %) and too little sample size to generalize to 75 %.

I know there is a "magic sample size" of about 30 questions that seems reasonable here. If I answer 30 questions and get 28 or 29 answers right then it will seem "reasonable" that I will get at least 65 % on the real test.

But how should I make a more exact and correct estimate? How many questions do I have to answer to generalize my mock score?

  • $\begingroup$ This is an interesting question! However, I don't understand the meaning of the first two sentences. What grade do you need to pass the test? 50%? $\endgroup$ – Michael R Jan 14 '17 at 19:24
  • $\begingroup$ Passing is 65 % $\endgroup$ – Niklas Jan 14 '17 at 19:26
  • $\begingroup$ The answer depends on both the sample size of the practice exam and the proportion of correctly answered question on the practice exam. $\endgroup$ – Michael R Jan 14 '17 at 20:49

First, we need to make two important assumptions: 1) all questions on a given exam are equal in terms of difficulty, and 2) the questions on the practice and real exams are equal in terms of difficulty.

Let $ X_i $ be a Bernoulli random variable that denotes whether or not the $ i_{th} $ answer on the practice exam is correct, i.e. $ X_i = 1 $ if the $ i_{th} $ answer is correct and $ X_i = 0 $ if the $ i_{th} $ answer is wrong.

Let $ p $ denote the proportion of correctly answered questions on the practice exam. Then $ \hat{p} $, the MLE of p, is equal to $ \hat{p} = \frac{\sum_{i=1}^{n}X_i}{n} = \bar{X}_{n} $.

Let $ Y $ denote the total number of correct answers on the real exam. Then $ Y \approx 80\hat{p} $. Since the minimum passing grade is 65%, we want to find:

$ P(Y\geq52) = P(80\hat{p} \geq 52) = P\big(\hat{p}\geq\frac{52}{80}\big) = P\big(\hat{p}\geq0.65\big) $

By the CLT, $ \frac{\hat{p}-E(\hat{p})}{\sqrt{Var(\hat{p})}} \overset{d}{\to} Z $. Furthermore, $ E(\hat{p}) = \bar{X}_n $ and $ Var(\hat{p}) = n\bar{X}_n(1-\bar{X}_n) $, which means that we get:

$ P\bigg(Z \geq \frac{0.65-\hat{p}}{\sqrt{Var(\hat{p})}} \bigg) = P\bigg(Z \geq \frac{0.65-\bar{X}_n}{\sqrt{n\bar{X}_n(1-\bar{X}_n)}} \bigg) $

In order to pass with probability $ (1-\alpha)100\% $, we need:

$ P\bigg(Z \geq Z_{\alpha} \bigg) = 1-\alpha $, which means that we need: $ Z_{\alpha} = \frac{0.65-\bar{X}_n}{\sqrt{n\bar{X}_n(1-\bar{X}_n)}} \rightarrow n = \Bigg(\frac{0.65-\bar{X}_n}{Z_{\alpha}\sqrt{\bar{X}_n(1-\bar{X}_n)}}\Bigg)^2 $

Note: in this formula, $ n $ is number of questions on the practice exam, while $ \bar{X}_n $ is the proportion of correctly answered questions on the practice exam.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.