# Probability of having a disease given a symptom is reported

A medical research team wishes to assess the usefulness of a certain symptom in the diagnosis of a particular disease. In a random sample of 775 patients with the disease, 744 reported having the symptom. In an independent random sample of 1380 subjects without the disease, 21 reported that they had the symptom. Find the predictive value positive and the predictive value negative for the syptom for the following hypothetical disease rates: 0.0001, 0.01, and 0.1

-
Hint: use Bayes' theorem. en.wikipedia.org/wiki/Bayes%27_theorem – Dan Brumleve Sep 12 '11 at 1:37

I'll give this a shot. I'm interpreting "predictive value" to mean the estimated probability that the disease is present, given that the symptom is (not) reported.

Let S be be event that the patient reports the symptom, and let D be the event that the patient has the disease.

Based on the data, we can estimate the probability of a patient reporting the symptom given that they have the disease as:

P(S | D) = 744/755

And we can estimate the probability of a patient reporting the symptom given that the patient does not have the disease as:

P(S | ¬D) = 21/1380

We know logically that P(S) = P(S ∧ D) + P(S ∧ ¬D), because these events are mutually exclusive. By Bayes' theorem we have:

P(S ∧ D) = P(D) * P(S | D)

P(S ∧ ¬D) = P(¬D) * P(S | ¬D) = (1 - P(D)) * P(S | ¬D)

And therefore:

P(S) = P(D) * 744/755 + (1 - P(D)) * 21/1380

P(S) = P(D) * 67391/69460 + 21/1380

Again by Bayes' theorem we have for the positive case:

P(D | S) = P(S | D) * P(D) / P(S)

P(D | S) = 68448 / (67391 + (1057 / P(D)))

And the negative case:

P(D | ¬S) = P(¬S | D) * P(D) / P(¬S)

P(D | ¬S) = (1 - P(S | D)) * P(D) / (1 - P(S))

P(D | ¬S) = 1012 / (-67391 + (68403 / P(D)))

Now for the given values of P(D), 0.0001, 0.01, and 0.01, we can compute the respective values of P(D | S): 22816/3656797, 22816/57697, and 22816/25987. Also we can compute the respective values of P(D | ¬S): 92/62178419, 92/615719, and 1012/616639.

-

As Dan Brumleve says, you need to use Bayes' theorem. With slightly different notation (using a bar for negation, and sometimes omitting the conjunction symbol, $AB \equiv A \wedge B$), if $S$ and $D$ are symptom and disease events/RVs, then Bayes' theorem tells us how to relate the probabilities of one in terms of the other or, more specifically, how to express the posterior probabilities of $D$ and $\overline{D}$ (conditioned on $S$ or $\overline{S}$) using the prior model probabilities (conditional probabilities of $S$ or $\overline{S}$ given $D$ or $\overline{D}$) and the priors of $D$ and $\overline{D}$. We are given the following (specially collected) normative data (counts), from which we can estimate our model's prior conditional probabilities, $$\matrix{ \color{green} {\#} & S & \overline{S} & \text{Total} \\~\\ D & 744 & 31 & 775 \\~\\ \overline{D} & 21 & 1359 & 1380 \\ } \qquad \qquad \matrix{ \color{green}{\approx} & P( S \mid \cdot ) & P(\overline{S} \mid \cdot ) \\~\\ P( \cdot \mid D ) & \tfrac{ 744}{ 775} & \tfrac{ 31}{ 775}\\~\\ P( \cdot \mid \overline{D}) & \tfrac{ 21}{1380} & \tfrac{1359}{1380} \\ }$$ where the two rows have been independently collected from sufficiently large samples as part of the evaluation of the test's reliability. In particular, they have been collected separately from each other, so that the relative row totals, or sample sizes, carry no information on (are not indicative of) the true prevalence of disease. The above counts can be used to estimate conditional model probabilities, i.e. the probabilities of $S$ or $\overline{S}$ given $D$ or $\overline{D}$, which are shown above on the right for easy comparison with the corresponding counts. $$\matrix{ P( S \mid D )\approx\tfrac{ 744}{ 775}=\tfrac{24}{25}\\ P(\overline{S} \mid D )\approx\tfrac{ 31}{ 775}=\tfrac{ 1}{25}\\ P( S \mid \overline{D})\approx\tfrac{ 21}{1380}=\tfrac{7}{460}\\ P(\overline{S} \mid \overline{D})\approx\tfrac{1359}{1380}=\tfrac{453}{460} }$$ We consider $S$ to be dependent on $D$, and $D$ to have (Bernoulli distribution and) prior probability (prevalence) $P(D)=p=10^{-n}$ for $n=1,2,4$ respectively. The (binary classification) test is how we predict $D$ after observing $S$ in a particular patient. Let P & N signify positive & negative symptom presence (or test results), and $+$ & $-$ signify actual disease presence or absence, respectively; the prediction is that ${D=S}$. Let us define true and false positives and negatives $\text{TP}=SD$, $\text{FP}=S\overline{D}$, $\text{FN}=\overline{S}D$, $\text{TN}=\overline{SD}$, and the related (simple) Hypothesis test errors (where the null hypothesis is $H_0=\overline{D}$) as follows: $$\matrix{ \text{event} &\quad& \text{probability/rate} & \quad & \text{a.k.a.}\\ \text{FP}|-= S \mid \overline{D} && \alpha && \text{type I error}\\ \text{TN}|-=\overline{S}\mid\overline{D}&& 1-\alpha &&\text{specificity}\\ \text{FN}|+=\overline{S}\mid D && \beta && \text{type II error}\\ \text{TP}|+= S \mid D && 1-\beta &&\text{sensitivity, power} }$$ The signs, $+=D$ and $-=\overline{D}$, above are actual disease states, which the "true" and "false" (correctly or incorrectly predicted) symptom states condition on. Having two sets of positive and negative states, one predicted or observed from a sample ($P=S,N=\overline{S}$), the other representing an actual or true state ($+=D,-=\overline{D}$), can be confusing; if so, using $S$ for symptom and $D$ for disease, and avoiding $P,N$ and their $T,F$ variants may prevent confusion. Hopefully the above clarifies their relationships, if & when a translation between the two notational perspectives is needed. Some quantities we are interested in (PPV, NPV), as well as some related quantities are: \eqalign{ \text{positive predictive value} & = P(D \mid S) = \frac{\text{TP}}{\text{FP}+\text{TP}} = \frac{\text{TP}}{\text{P}} = \text{PPV} \\ \text{negative predictive value} & = P(\overline{D} \mid \overline{S}) = \frac{\text{TN}}{\text{FN}+\text{TN}} = \frac{\text{TN}}{\text{N}} = \text{NPV} \\ \text{sensitivity} &= P(S \mid D) = \frac{\text{TP}}{\text{FN}+\text{TP}} = \frac{\text{TP}}{+} = 1 - \beta \\ \text{specificity} &= P(\overline{S} \mid \overline{D}) = \frac{\text{TN}}{\text{FP}+\text{TN}} = \frac{\text{TN}}{-} = 1 - \alpha \\ \text{type I error} &= P(S \mid \overline{D}) = \frac{\text{FP}}{\text{FP}+\text{TN}} = \frac{\text{FP}}{-} = \alpha \\ \text{type II error} &= P(\overline{S} \mid D) = \frac{\text{FN}}{\text{FN}+\text{TP}} = \frac{\text{FN}}{+} = \beta \\ } For notational simplicity, I am identifying events with counts and probabilities with rates. However, it must be noted that each of the ratios above is only valid when the pair of counts of true/false positives/negatives are "commensurate", or "calibrated", in the sense that they come from the same (population) sample, or at least reflect the true population prevalence levels; otherwise, they introduce bias. For example, we cannot use it to calculate PPV$=\frac{TP}{TP+FP}=\frac{P(SD)}{P(SD)+P(S\overline{D})}$ as $\frac{744}{744+21}$. This is because the first two rows have different sample sizes, and the calculation above fails to "normalize" the values first. Replacing $21$ & $744$ in this calculation by the fractions in which they appear as denominators above gets us halfway there, but we must also normalize these fractions by their prevalence, weighing them by how often they should occur, to correct the calculation. In fact, PPV therefore depends on the unknown prevalence $p$, since the joint probabilites below can be calculated by conditioning on $D$: \eqalign{ P(SD)&=P(S|D)\cdot P(D)=(1-\beta)p\approx\tfrac{24}{25}p \\ P(S\overline{D})&=P(S|\overline{D}) \cdot P(\overline{D})=\alpha(1-p) \approx\tfrac{7}{460}(1-p) } so that $$\text{PPV}=\frac{P(SD)}{P(SD)+P(S\overline{D})} \approx\frac{\frac{24}{25}p}{\frac{24}{25}p+\frac{7}{460}(1-p)} =\frac{p}{p+\frac{35}{2208}(1-p)} =\frac{2208p}{2173p+35},$$ which are estimations based on our two samples. In our case, this means that we can use the last two formulas to estimage the sensitivity and specificity, since we were given data conditioned on $D$ -- precisely what we need to evaluate the usefulness of the test.

So we can summarize the entire information content of the above data by saying that they give us these estimates of the type I and II error rates, where the larger, unreduced denominators represent the respective sample sizes of "true" disease states used in the estimation: $$\alpha\approx\tfrac{21}{1380}=\tfrac{7}{460} \qquad\text{and}\qquad \beta\approx\tfrac{31}{775}=\tfrac{1}{25}.$$

Bayes's theorem "calibrates" the reciprocal conditionals (conditioned on $S$) in the first two formulas above for positive and negative predicted value, along with their negations, by explicitly using the model parameter, the actual (but unknown) prevalence $p=P(D)$: \eqalign{ P(D \mid S) & = \frac{P(S \mid D) \cdot P(D)}{P(S \mid D) \cdot P(D) + P(S \mid \overline{D}) \cdot P(\overline{D})} = \frac{(1-\beta) \, p}{(1-\beta) \, p + \alpha \, (1-p)} = \frac{1}{1+\frac{\alpha}{1-\beta}\cdot\frac{1-p}{p}} \\ P(\overline{D} \mid S) & = \frac{P(S\mid\overline{D}) \cdot P(\overline{D})}{P(S\mid D) \cdot P(D) + P(S \mid \overline{D}) \cdot P(\overline{D})} = \frac{\alpha \, (1-p)}{(1-\beta) \, p + \alpha \, (1-p)} = \frac{1}{1+\frac{1-\beta}{\alpha}\cdot\frac{p}{1-p}} \\ P(D \mid \overline{S}) & = \frac{P(\overline{S}\mid D) \cdot P(D)}{P(\overline{S}\mid D) \cdot P(D) + P(\overline{S} \mid \overline{D}) \cdot P(\overline{D})} = \frac{\beta \, p}{\beta \, p + (1-\alpha) \, (1-p)} = \frac{1}{1+\frac{1-\alpha}{\beta}\cdot\frac{1-p}{p}} \\ P(\overline{D} \mid \overline{S}) & = \frac{P(\overline{S} \mid \overline{D}) \cdot P(\overline{D})}{P(\overline{S} \mid D) \cdot P(D) + P(\overline{S} \mid \overline{D}) \cdot P(\overline{D})} = \frac{(1-\alpha) \, (1-p)}{\beta \, p + (1-\alpha) \, (1-p)} = \frac{1}{1+\frac{\beta}{1-\alpha}\cdot\frac{p}{1-p}} \\ } The representations with odds ratios on the far right hand sides above shows the algebraic-functional dependence more clearly. The ratios $\tfrac\alpha{1-\beta}\approx\frac{35}{2208}$ and $\tfrac\beta{1-\alpha}\approx\frac{92}{2265}$ are "not quite" odds ratios, since they "mix" $\alpha$ & $\beta$ asymmetrically; they are, in fact, called positive and negative likelihood ratios, $\text{LR}+=\frac{1-\beta}\alpha$ & $\text{LR}-=\frac\beta{1-\alpha}$. If we use square brackets to write $y=\left[x\right]=\frac{x}{1-x}$ for the odds ratio of a probability $x$ (an invertible function mapping $[0,1]\to[0,\infty]$ with interesting symmetry properties and inverse $x=\frac{y}{1+y}=\frac{1}{1+1/y}$, as seen in the form of the right hand sides above) and $D^\delta=D,\overline{D}$ for $\delta=+1,-1$ respectively (both nonstandard notations), these become even more succinct: \eqalign{ \left[P(D^\delta \mid S)\right] &= \left( \tfrac\alpha{1-\beta} / \left[p\right] \right)^{-\delta} &= \left( \left[p\right] \text{LR}+ \right)^{\delta} \\ \left[P(D^\delta \mid \overline{S})\right] &= \left( \tfrac\beta{1-\alpha} \cdot \left[p\right] \right)^{\delta} &= \left( \left[p\right] \text{LR}- \right)^{\delta} } These are also related to the diagnostic odds ratio (using square brackets for odds ratios again): $$\text{DOR} =\frac{\text{LR}+}{\text{LR}-} =[\text{PPV}][\text{NPV}] =[\text{sensitivity}][\text{specificity}] =\frac{1-\alpha}{\alpha}\cdot\frac{1-\beta}{\beta} \approx\tfrac{10872}{7} \text{.}$$ In any case, this gives us \eqalign{ \text{PPV}&=P(D \mid S) =\frac{1}{1+\frac{\alpha}{1-\beta}\cdot\frac{1-p}{p}} \approx \frac{2208p}{2173p+35} \\ \text{NPV}&=P(\overline{D} \mid \overline{S}) =\frac{1}{1+\frac{\beta}{1-\alpha}\cdot\frac{p}{1-p}} \approx \frac{2265(p-1)}{2173p-2265} =\frac{1-p}{1-\frac{2173}{2265}p} } which is plotted versus prevalence $p$ and log prevalence $n=-\log p$ with a table for relevant $n$ below using sage.

Looking at the graphs, we see that the tradeoff between PPV & NPV accuracy is asymmetric. PPV increases with prevalence and is unreliable for low $p$ (near $0$), while NPV decreases with prevalence and unreliable for high $p$ (near unity). In particular, NPV is good (near $1$) for $n=1,2,4$ but PPV only marginally achieves this for $n=1$. Based on our estimated priors, we can calculate that PPV$\approx$NPV for $p=\frac{920\sqrt{2114}-26425}{41287}\approx0.3845034823$ ($n\approx0.4151$), shown with dotted grey lines.

One standard way of quantifying this information is using ROC curves, but others also exist. In fact, estimating PPV and NPV in unknown populations is an interesting problem.

-