1
$\begingroup$

I'm having a hard time trying to undestand this.

I'm doing an exercise where I have to formalise 'your test has a false-positive rating of 5%'. If $B$ means that the test is positive, and $A$ means that a given person has the disease, then is $P(A^{C}|B)=5\%$, or $P(B|A^{C})=5\%$?

Intuitively, I would have picked the second option. A false-positive is when I'm testing a person who doesn't have the disease, but still get a positive result. However, this website http://vassarstats.net/bayes.html, says otherwise. According to it, a false positive would mean the first.

I don't get this. Could someone please elaborate.

Edit: I thought that after reading Bram28's answer I understood it, but actually I'm no less confused than before. For reference, I am referring to exercise 3.6 on page 19 in this book: http://www.karlin.mff.cuni.cz/~lachout/Vyuka/O-Sem/JacodProtter2004.pdf. According to Bram28's answer, false positive means $P(A^{C}|B)$. But this fixes $P(A|B)=1-P(A^{C}|B)$, which is what is asked to be determined in the question, without needing the 'accuracy' which is also given in the exercise. Does somebody understand what is happening there?

$\endgroup$
2
$\begingroup$

Quasar gave you the correct way to calculate what you wanted to calculate, but I understand you are looking to clear up some of your confusion regarding terminology, so I hope the following helps (but I'll give you that it is all very confusing though!!):

A 'false positive' occurs when the disease is not present even though the test is positive. So, this is the event $B \land A^C$.

So, if you want to know the probability of getting a false positive, that is $P(B \land A^C)$, and that we can work out in two different ways:

$P(B \land A^C) = P(B)*P(A^C|B)$ or

$P(B \land A^C) = P(A^C)*P(B|A^C)$

Here:

$P(A^C|B)$ is the probability of the test being wrong (i.e. being a false positive) when the test comes out positive. So: out of all the cases where we test someone and they test positive, how often is that test result the wrong result?

$P(B|A^C)$ is the probability of the test getting the wrong result (and again be a false positive) given that the person tested does not have the disease.

Now, the confusing thing is that one can reasonably use the phrase 'false positive rate' to mean any of these three different probabilities: $P(B \land A^C)$ (how often do false positives occur?), $P(A^C|B)$ (how often is a positive test false?), and $P(B|A^C)$ (how often do we get a false positive for a healthy being?)

Something similar goes for the term 'accuracy'. That is, we may say that some test is '95% accurate', but what does that mean? Here, we could reasonably mean (or at least: the 'person on the street' could reasonably interpret this as):

$P(B|A)$: How often does the test accurately diagnose someone with the disease?

$P(A|B)$: if I test positive, what is the chance that I actually have that disease? (this could be called a kind of 'predictive accuracy')

Now, professionals will typically mean the first one (but it never hurts to ask them and be clear about what they really mean when they use this phrase!!!), because $P(B|A)$ is much more 'stable' than $P(A|B)$, and that is because if we compare a situation where a large percentage of the population has the disease with a situation where a small percentage of the population has the disease (in other words, as $P(A)$ changes over time ... which that can of course happen just fine), then $P(A|B)$ can change quite a bit as well, while this is far less true for $P(B|A)$: the chance for a test to get the correct result when we test someone who has the disease will be pretty much the same over time (unless we suppose that our biophysical states change significantly over time, which is far less likely).

So, if we say that a test is "95% accurate", we are probably referring to $P(B|A) = 0.95$.

However, here is one more wrinkle:

If 'the test is 95% accurate" means $P(B|A) = 0.95$, then do we know $P(B^C|A^C)$, i.e. the 'accuracy' of correctly diagnosing (i.e. the test coming out negative) a person that does not have the disease? No, we don't. Maybe the test is on the 'conservative' side, and will more likely come out positive for a person without the disease, than that it will come out negative for a person with the disease (indeed, in this context, a false negative could have far more dire consequences than a false positive!). So, in that case, $P(B^C|A^C)$ will be smaller than 95%. In fact, if you think about it, one could try and define an 'accuracy' rating that takes into account both of these probabilities, i.e. one could also reasonably interpret an 'accuracy rating of 95%' as $P(test correct)=0.95$ where:

$P(test correct)= P(B|A)*P(A) + P(B^C|A^C)*P(A^C)$

In the exercise in the book you refer to, though, they do specify a 'false positive rating' in addition to the accuracy rating, and by the false positive rating they mean $P(B|A^C)$. Which, by an analogous argument to the one given above, makes sense: $P(B|A^C)$ is likely to be more stable over time than $P(A^C|B)$.

The take home message is this though: the terminology is confusing, sine it can be interpreted in many reasonable, but different ways. Indeed, I assume the professionals themselves have a hard time getting all these distinctions straight. Therefore, it never hurts to ask what exactly they mean: statistical confusion has probably hurt more people than any one particular disease itself!

$\endgroup$
  • $\begingroup$ This was a typo on my side. But why is test-accuracy defined as $P(B|A)$ then? $\endgroup$ – azureai Jan 26 '17 at 1:52
  • $\begingroup$ @see That's an excellent question! Now, I don't see accuracy defined on that website, but I do know that the 'accuracy' of a test can mean different things: Sometimes they mean $P(A|B)$ (called 'predicate accuracy', i.e. Given that I test positive, what is the chance I actually have the disease?), but it is (more commonly) used to mean $P(B|A)$, i.e. out of all the cases where the disease is present, how often dows the test detect that, i.e. how well does the test classify the diseases? So yes, it's all rather confusing!!! $\endgroup$ – Bram28 Jan 26 '17 at 2:04
  • $\begingroup$ @see there is an old puzzle that goes like this: 'suppose I test positive for a disease, and that the test is 95% accurate. What is the chance I have the disease?'. Our immediate reaction is '95% of course!'. And it would indeed be 95% if by 'accuracy' we mean $P(A|B)$. But again, we typically mean by 'test-accuracy' $P(B|A)$. ... And now it turns out that the chance of me having the disease depends on (not surprisingly) what percentage of the population has the disease, i.e. $P(A)$: if that is only a very small percentage, then there is a good chance that I am one of those false positives! $\endgroup$ – Bram28 Jan 26 '17 at 2:12
  • $\begingroup$ @see How does your exercise define A and B? Are they maybe swapped from the website? Anyway, the answer to the puzzle is not the 95% that many people think it is. It really depends on the 'base rate' of the disease. example: suppose there are 1000 people and 2%, i.e. 20 people have the disease, and so 980 do not. Now, when we then say that the test is 95% accurate, we mean that 95%, i.e. 19 of the 20 diseased people test positive, and that 95% of the 980, i.e. 931 non-diseased people correctly test negatively. but this means that 49 non-diseased people test positive. ...(coninued) $\endgroup$ – Bram28 Jan 26 '17 at 2:15
  • $\begingroup$ Yeah this is exactly the exercise I am doing, and it got me indeed puzzled which is why I looked up what false-positive actually means, leading to this post. It's just counter-intuitive to me why one would chose as test-accuracy $P(B|A)$, everything else became clear from your answer. $\endgroup$ – azureai Jan 26 '17 at 2:23
2
$\begingroup$

@see,

We treat the test as evidence and the occurrence of the disease in the population as the hypothesis. And in problems like these, we are given $P(evidence|hypothesis)$. Therefore, $P(B|A^C)=0.99$. For ease, I define:

$T:= \text{Test returns positive}$

$D:= \text{Individual has the disease}$

We are given:

True positive : $P(T|D)=0.99$, and $P(T^C|D)=0.01$

False positive(Test falsely returns a positive, when in fact the person does not have the disease; 500 test positive in 10000 people not having the disease): $P(T|D^C)=0.05$ and $P(T^C|D^C) = 0.95$

And the occurrence of the disease is rare. $P(D)=1/10000=0.0001$

We are being asked, that if the test returns positive(evidence), what is the likelihood that you have AIDS(hypothesis)? $P(D|T)=?$

Using Bayes rule and LOTP,

$$\begin{align} P(D|T)&=\frac{P(T|D)P(D)}{P(T)}\\ &=\frac{P(T|D)P(D)}{P(T|D)P(D)+P(T|D^C)P(D^C)}\\ &=\frac{(0.99)(0.0001)}{(0.99)(0.0001)+(0.05)(0.9999)}\\ &=0.001976285 \end{align}$$

Intuitively, in a population of $10,000$, $1$ is likely to have the disease, $9999$ are likely not to have the disease. The test has an accuracy rate of $99\%$. Given that they have the disease, $0.99\cdot{1}=0.99$ persons are likely to test positive, $0.01\cdot{1}=0.01$ persons would falsely test negative. Given that they do not have the disease, $9499.05$ persons would test negative, $499.95$ persons would falsely test positive.

We are asked, given that you test positive, what is the likelihood you have AIDS? So, that would be $0.99/(0.99+499.95)=0.001976285$.

Hope that answers your question.

$\endgroup$
  • 2
    $\begingroup$ Isn't your definition of false-positive the exact opposite of Gram28's? $\endgroup$ – azureai Jan 26 '17 at 4:02
  • $\begingroup$ Modified the answer for the accuracy rate=$0.99$. From what I understand, $T|D=\text{True positive}$, $T^C|D=\text{False Negative}$, $T|D^C=\text{False Positive}$, $T^C|D^C=\text{True Negative}$ $\endgroup$ – Quasar Jan 26 '17 at 4:09
  • $\begingroup$ Also, intuitively, think how would you deduce the true-positivity and false-positivity of a diagnostic test? Conduct trials on a large population. How many test positive, restricting our attention to diseased people? That's, $T|D$ - True positive. How many test positive, restricting our attention to the healthy people? $T|D^C$ - False positive. Does that help? $\endgroup$ – Quasar Jan 26 '17 at 4:15
  • 1
    $\begingroup$ Your definition matches my intuition, as I wrote in my opening post. Your solution also matches what I calculated on my own when I first did the exercise, so I guess I'll go with this interpretation. $\endgroup$ – azureai Jan 26 '17 at 4:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.