# Probability of having a disease given a symptom is reported

A medical research team wishes to assess the usefulness of a certain symptom in the diagnosis of a particular disease. In a random sample of 775 patients with the disease, 744 reported having the symptom. In an independent random sample of 1380 subjects without the disease, 21 reported that they had the symptom. Find the predictive value positive and the predictive value negative for the syptom for the following hypothetical disease rates: 0.0001, 0.01, and 0.1

-
Hint: use Bayes' theorem. en.wikipedia.org/wiki/Bayes%27_theorem –  Dan Brumleve Sep 12 '11 at 1:37

I'll give this a shot. I'm interpreting "predictive value" to mean the estimated probability that the disease is present, given that the symptom is (not) reported.

Let S be be event that the patient reports the symptom, and let D be the event that the patient has the disease.

Based on the data, we can estimate the probability of a patient reporting the symptom given that they have the disease as:

P(S | D) = 744/755

And we can estimate the probability of a patient reporting the symptom given that the patient does not have the disease as:

P(S | ¬D) = 21/1380

We know logically that P(S) = P(S ∧ D) + P(S ∧ ¬D), because these events are mutually exclusive. By Bayes' theorem we have:

P(S ∧ D) = P(D) * P(S | D)

P(S ∧ ¬D) = P(¬D) * P(S | ¬D) = (1 - P(D)) * P(S | ¬D)

And therefore:

P(S) = P(D) * 744/755 + (1 - P(D)) * 21/1380

P(S) = P(D) * 67391/69460 + 21/1380

Again by Bayes' theorem we have for the positive case:

P(D | S) = P(S | D) * P(D) / P(S)

P(D | S) = 68448 / (67391 + (1057 / P(D)))

And the negative case:

P(D | ¬S) = P(¬S | D) * P(D) / P(¬S)

P(D | ¬S) = (1 - P(S | D)) * P(D) / (1 - P(S))

P(D | ¬S) = 1012 / (-67391 + (68403 / P(D)))

Now for the given values of P(D), 0.0001, 0.01, and 0.01, we can compute the respective values of P(D | S): 22816/3656797, 22816/57697, and 22816/25987. Also we can compute the respective values of P(D | ¬S): 92/62178419, 92/615719, and 1012/616639.

-

As Dan Brumleve says, you need to use Bayes' theorem. With slightly different notation (using a bar for negation, and sometimes omitting the conjunction symbol, $AB \equiv A \wedge B$), if $S$ and $D$ are symptom and disease events/RVs, then Bayes' theorem tells us how to relate the probabilities of one in terms of the other or, more specifically, how to express the posterior probabilities of $D$ and $\overline{D}$ using model probabilities (conditional probabilities of $S$ conditioned on $D$) and the priors of $D$ and $\overline{D}$. We are given the following (specially collected) normative data $$\matrix{ \color{green} {\#_D} & S & \overline{S} & \text{Total} \\~\\ D & 744 & 31 & 775 \\~\\ \overline{D} & 21 & 1359 & 1380 \\~\\ \text{Total} & 765 & 1390 & 2155 \\ } \qquad \qquad \matrix{ \color{green}{\approx} & P( S \mid \cdot ) & P(\overline{S} \mid \cdot ) \\~\\ P( \cdot \mid D ) & \tfrac{ 744}{ 775} & \tfrac{ 31}{ 775}\\~\\ P( \cdot \mid \overline{D}) & \tfrac{ 21}{1380} & \tfrac{1359}{1380} \\ }$$ where the two rows have been independently collected from sufficiently large samples as part of the evaluation of the test's reliability. In particular, they have been collected separately from each other, so that the row totals carry no information on the true prevalence of disease. The above counts can be used to estimate conditional model probabilites i.e. the probabilities of $S$ or $\overline{S}$ given $D$ or $\overline{D}$, which are shown above on the right for easy comparison with the corresponding counts. $$\matrix{ P( S \mid D ) \approx \tfrac{ 744}{ 775} \\ P(\overline{S} \mid D ) \approx \tfrac{ 31}{ 775} \\ P( S \mid \overline{D}) \approx \tfrac{ 21}{1380} \\ P(\overline{S} \mid \overline{D}) \approx \tfrac{1359}{1380} }$$ We consider $S$ to be dependent on $D$, and $D$ to have (Bernoulli distribution and) prior probability (prevalence) $P(D)=p=10^{-n}$ for $n=1,2,4$ respectively. The (binary classification) test is how we predict $D$ after observing $S$ in a particular patient. Tthe test result is positive (P) when the symptom is present, i.e. $S$ is observed, and negative (N) when $\overline{S}$ is observed; the prediction is that ${D=S}$). We define true and false positives and negatives, and the related (simple) Hypothesis test errors (where the null hypothesis is $H_0=\overline{D}$), as $$\matrix{ \text{event} &\quad& \text{probability/rate} & \quad & \text{a.k.a.}\\ \text{FP}|-= S \mid \overline{D} && \alpha &&\text{type I error}\\ \text{TP}|+= S \mid D && 1-\alpha &&\text{sensitivity} \\ \text{FN}|+=\overline{S}\mid D && \beta &&\text{type II error}\\ \text{TN}|-=\overline{S}\mid \overline{D}&& 1-\beta &&\text{specificity, power} }$$ In the above, $+$ and $-$ are to be interpreted as the actual positive and negative probabilities, counts or events, rather than the "true" or "false" (correctly or incorrectly predicted) counts. However, having two sets of positives and negatives, one predicted or observed from a sample, and the other representing an actual or true state, can lead to confusion. $S$ for symptom and $D$ for disease is much less confusing. Some quantities we are interested in (PPV, NPV), as well as some related quantities are: \eqalign{ \text{positive predictive value} & = P(D \mid S) = \frac{\text{TP}}{\text{FP}+\text{TP}} = \frac{\text{TP}}{\text{P}} = \text{PPV} \\ \text{negative predictive value} & = P(\overline{D} \mid S) = \frac{\text{TN}}{\text{FN}+\text{TN}} = \frac{\text{TN}}{\text{N}} = \text{NPV} \\ \text{sensitivity} &= P(S \mid D) = \frac{\text{TP}}{\text{FN}+\text{TP}} = \frac{\text{TP}}{+} = 1 - \alpha \\ \text{specificity} &= P(\overline{S} \mid \overline{D}) = \frac{\text{TN}}{\text{FP}+\text{TN}} = \frac{\text{TN}}{-} = 1 - \beta \\ } For notational simplicity, I am identifying events with counts and probabilities with rates. However, it must be noted that each of the ratios above is only valid when the pair of counts of true/false positives/negatives are "commensurate", or "calibrated", in the sense that they come from the same (population) sample, or at least reflect the true population prevalence levels; otherwise, they introduce bias. For example, we cannot use it to calculate PPV as $\frac{744+21}{775+1380}$. In fact, PPV$=\frac{744}{775}p+\frac{21}{1380}(1-p)$ depends on the unknown prevalence. In our case, this means that we can use the last two formulas to get the sensitivity and specificity, since we were given data conditioned on $D$ -- precisely what we need to evaluate the usefulness of the test.

So we can summarize the entire information content of the above data by saying that they give us $$\alpha=\frac{21}{1380}=\frac{7}{460} \qquad\text{and}\qquad \beta=\frac{31}{775}=\frac{1}{25}.$$

Bayes's theorem "calibrates" the reciprocal conditionals (conditioned on $S$) in the first two formulas above, for positive and negative predicted value, by explicitly using the model parameter, the actual (but unknown) prevalence $p=P(D)$: \eqalign{ P(D \mid S) & = \frac{P(S \mid D) \cdot P(D)}{P(S \mid D) \cdot P(D) + P(S \mid \overline{D}) \cdot P(\overline{D})} = \frac{(1-\alpha) \, p}{(1-\alpha) \, p + \alpha \, (1-p)} = \frac{1}{1+\frac{\alpha}{1-\alpha}\cdot\frac{1-p}{p}} \\ P(\overline{D} \mid S) & = \frac{P(S \mid \overline{D}) \cdot P(\overline{D})}{P(S \mid D) \cdot P(D) + P(S \mid \overline{D}) \cdot P(\overline{D})} = \frac{\alpha \, (1-p)}{(1-\alpha) \, p + \alpha \, (1-p)} = \frac{1}{1+\frac{1-\alpha}{\alpha}\cdot\frac{p}{1-p}} \\ P(D \mid \overline{S}) & = \frac{P(\overline{S} \mid D) \cdot P(D)}{P(\overline{S} \mid D) \cdot P(D) + P(\overline{S} \mid \overline{D}) \cdot P(\overline{D})} = \frac{\beta \, p}{\beta \, p + (1-\beta) \, (1-p)} = \frac{1}{1+\frac{1-\beta}{\beta}\cdot\frac{1-p}{p}} \\ P(\overline{D} \mid \overline{S}) & = \frac{P(\overline{S} \mid \overline{D}) \cdot P(\overline{D})}{P(\overline{S} \mid D) \cdot P(D) + P(\overline{S} \mid \overline{D}) \cdot P(\overline{D})} = \frac{(1-\beta) \, (1-p)}{\beta \, p + (1-\beta) \, (1-p)} = \frac{1}{1+\frac{\beta}{1-\beta}\cdot\frac{p}{1-p}} \\ } The representations with odds above shows the algebraic-functional dependence more clearly. If we write $\left[p\right]=\frac{p}{1-p}$ for the odds of $p$ (an invertible function with interesting symmetry properties) and $D^\delta=D,\overline{D}$ for $\delta=+1,-1$ respectively (a nonstandard notation!), these become even more succinct: \eqalign{ \left[P(D^\delta \mid S)\right] &= \left( \left[\alpha\right]/ \left[ p \right] \right)^{ \delta} \\ \left[P(D^\delta \mid \overline{S})\right] &= \left( \left[\beta\right]\cdot \left[ p \right] \right)^{-\delta} } Perhaps this hints at a relation to diagnostic odds ratios. This gives us \eqalign{ \text{PPV}&=P(D \mid S) =\frac{1}{1+\frac{\alpha}{1-\alpha}\cdot\frac{1-p}{p}} \approx \frac{453p}{446p+7} \\ \text{NPV}&=P(\overline{D} \mid S) =\frac{1}{1+\frac{1-\alpha}{\alpha}\cdot\frac{p}{1-p}} \approx \frac{7(1-p)}{446p+7} } which is plotted below using sage.

Looking at the graph, one word should enter our minds... One standard way of quantifying this is using ROC curves, but others also exist. In fact, estimating PPV and NPV in unknown populations is an interesting problem.

-