So I'm curious about something relevant to me in real life, but it's been forever since I took prop/stats so I don't remember how to calculate it. The accuracy of a COVID test is 70%. Like, if you have the virus, there is a 70% chance the test will come back positive, but a 30% chance you'll get a false negative. If you don't have the virus, there is a 100% chance it will come back negative (I don't think this is strictly true, but false positives with these tests are unlikely, so I'm ignoring them, and I'm fairly certain it's not relevant to this question). I'm almost certain if I had the virus, I would have transmitted it to my girlfriend. Let's assume for the sake of this problem that that's true. If I had the virus, she had it as well. We each took a test, and they each came back negative. What is the probability my negative result is accurate given this information, instead of if I was the only one who had taken the test?

  • 1
    $\begingroup$ Use Bayes theorem. Read the section on Wikipedia that uses the alternative version of Bayes en.wikipedia.org/wiki/Bayes%27_theorem . The example they give for drug testing is precisely the same calculation that you want to do. $\endgroup$ – Quotable Jun 9 '20 at 5:25
  • $\begingroup$ @Quotable Does not one need for this to know the "objective" probability to have the virus? $\endgroup$ – user Jun 9 '20 at 5:52
  • $\begingroup$ I already know I dont have it, I'm fine and this test was over a month ago. Im not trying to be exact, just get an approximate idea of how much more accurate two tests with the same result are than just one $\endgroup$ – PCRevolt Jun 9 '20 at 5:54

First, define some notation. Suppose we perform a series of independent tests, each with the same operating characteristics. That is to say, they are independent in the sense that the outcome of one test does not depend on the outcome of any previous tests; and they have the same operating characteristics in the sense that the probabilities of true/false positives/negatives are the same (as opposed to two different types of tests with different reliabilities).

Let $T_i$ be the event that the $i^{\rm th}$ test result is positive, and $\bar T_i$ be the complementary event that the $i^{\rm th}$ test result is negative. Let $C$ represent the event that the person tested actually has the condition, and $\bar C$ represent the complementary event that the person tested does not have the condition. We assume for the sake of simplicity that any individual's disease status is fixed for all tests conducted.

First, we model the scenario in which a person is tested once. You give the following probabilities for the test's operating characteristics:

$$\Pr[T_i \mid C] = 0.7 \\ \Pr[\bar T_i \mid C] = 0.3 \\ \Pr[T_i \mid \bar C] = 0 \\ \Pr[\bar T_i \mid \bar C] = 1. $$

At this point, we should note that we lack required information to perform any meaningful inference: in particular, we lack $\Pr[C]$, the prevalence of disease in the population. Even if we had $\Pr[T_i]$, we could recover prevalence via $$\Pr[T_i] = \Pr[T_i \mid C]\Pr[C] + \Pr[T_i \mid \bar C]\Pr[\bar C] = 0.7 \Pr[C].$$ Even if $\Pr[T_i \mid \bar C] > 0$, say some small number $\epsilon$, we can still get prevalence since the above becomes $$\Pr[T_i] = 0.7 \Pr[C] + \epsilon (1 - \Pr[C]).$$ But without either marginal probability $\Pr[T_i]$ or $\Pr[C]$, the desired probability is not uniquely determined.

To see this concretely, consider the probability of not being infected given a single negative test: $$\Pr[\bar C \mid \bar T_1] = \frac{\Pr[\bar T_1 \mid \bar C]\Pr[\bar C]}{\Pr[\bar T_1]} = \frac{\Pr[\bar C]}{0.3 \Pr[C] + \Pr[\bar C]} = \frac{1}{0.3/\Pr[\bar C] + 0.7}.$$ This is a function of the disease prevalence; so if half the population is infected, the posterior probability of not being infected given a negative result is $10/13 \approx 77\%$; but if only $3\%$ of the population is infected, the probability of not being infected given a negative result is much higher, over $99\%$.

With this in mind, let's now turn our attention to a scenario where a person is tested twice, and both tests are negative. Moreover, let us also use $\Pr[T_i \mid \bar C] = \epsilon$ as discussed above. We have $$\begin{align*}\Pr[\bar C \mid \bar T_1 \bar T_2] &= \frac{\Pr[\bar T_1 \bar T_2 \mid \bar C]\Pr[\bar C]}{\Pr[\bar T_1 \bar T_2]} \\ &= \frac{\Pr[\bar T_1 \mid \bar C] \Pr[\bar T_2 \mid \bar C]\Pr[\bar C]}{\Pr[\bar T_1 \mid C] \Pr[\bar T_2 \mid C]\Pr[C] + \Pr[\bar T_1 \mid \bar C] \Pr[\bar T_2 \mid \bar C]\Pr[\bar C]} \\ &= \frac{(1-\epsilon)^2 \Pr[\bar C]}{(0.3)^2 (1 - \Pr[\bar C]) + (1 - \epsilon)^2 \Pr[\bar C]} . \end{align*}$$ Now we can play with some hypothetical numbers. Say the true prevalence is $3\%$, and $\epsilon = 10^{-5}$, i.e. the false positive rate is $1$ in $100000$ tests. Then $$\Pr[\bar C \mid \bar T_1 \bar T_2] \approx 0.997224.$$ This is effectively no different than if $\epsilon = 0$. Also contrast this with the single-test scenario, which had a posterior probability of about $0.9908$. With a repeat test, we gain slightly more confidence that the person tested is in fact not infected.

But does this remain true if the population prevalence is different? There can be no doubt that multiple negative results increases the probability of not being infected, but by how much, and does this depend on the prevalence? If $\Pr[C] = 0.1$, we find that a single test gives us $$\Pr[\bar C \mid \bar T_1] \approx 0.967742,$$ but $$\Pr[\bar C \mid \bar T_1 \bar T_2] \approx 0.990099.$$ So we see that as the disease prevalence increases, the value of repeated negative tests as they relate to diagnostic accuracy also increases.

In closing, it is worth mentioning that we can model more sophisticated scenarios, such as not assuming that transmission occurs with certainty prior to the second test. But this introduces another variable for which an assumption must be made.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.