# Bayesian classifier to predict user behavior

I am trying to find a way to calculate the probability that a user will pass a test at a certain certification authority, based on the user's history with answering questions using our service. Looking at a single question at a time (I know how to combine the probabilities), I have the following data:

• pass - The percentage of users who passed the test at the certification authority on their first attempt who answered the question correctly.
• fail - he percentage of users who failed the test at the certification authority on their first attempt who answered the question correctly.
• correct - if the current user answered the question correctly.

Given an example question:

$pass = 0.96$

$fail = 0.42$

If the user answered the question correctly, I believe the probability that he will pass (given that question as the only evidence), is something like:

$P(pass) = \frac{0.96}{(0.96 + 0.42)} = 0.697$

Am I correct? What if the user answered the question wrongly the first time?

On average, 82% of our candidates pass the certification on the first attempt. Of everyone who tries to take the certification, only 50% passes.

Edit: What about a second question, where 70% of candidates who pass the test on their first attempt answer correctly, but 80% of candidates who fail the test on their first attempt answer correctly? (meaning $pass = 0.7$ and $fail = 0.8$) Answering this question incorrectly will contribute to passing the test, even though it logically shouldn't.

-

You are missing some information, for example what proportion of candidates pass the test on the first attempt (though it might be something else like the probability that this question is answered correctly). You also have some notation issues: pass seems to have two different meanings.

For illustration, let's suppose the proportion of candidates who pass the test on the first attempt is 0.8.

Then if the user answered the question correctly (on a first attempt), the probability that he will pass overall (on a first attempt) is $$\frac{0.8 \times 0.96}{0.8 \times 0.96 + (1-0.8) \times 0.42} \approx 0.901$$ while if the user answered the question incorrectly (on a first attempt), the probability that he will pass overall (on a first attempt) is $$\frac{0.8 \times (1-0.96)}{0.8 \times (1-0.96) + (1-0.8) \times (1-0.42)} \approx 0.216$$

The answers are sensitive to the 0.8 illustration. If instead the value had been 0.2 then the two conditional probabilities would have been about 0.364 and 0.017.

Your value of 0.697 (or 0.696) corresponds to the case where the overall pass rate for the whole test is 0.5; in this case the second conditional probability would be about $\frac{1-0.96}{(1-0.96) + (1-0.42)} \approx 0.0.065$. In the same situation, your 0.304 is the probability that the candidate will fail overall given a correct answer to this particular question. But you should make the 0.5 explicit and include it in the fractions.

-
Thank you. I did have the fraction (0.82 in my case, but a good guess on your end) in my calculations, but removed it because of the skewed results. I am getting some data I did not expect, though; consider the question were 70% of users who pass answer correctly, but 80% of users who fail answer correctly. If you answer this question wrongly, you will actually be more likely to pass according to this algorithm? Am I misunderstanding something? –  Vegard Larsen Mar 7 '11 at 13:08
No - just that it is a peculiar question. Perhaps it is easy to answer correctly using a bad method which fails in other contexts. Or perhaps the rest of the test is more theoretical while this question is more practical, so practical people find it easier than others do, but find the rest of the test harder. –  Henry Mar 7 '11 at 14:49