I'm trying to understand the law of total probability as it relates to discrete random variables.

So, I know that the law of total probability means

$Pr[X=x]=\sum_y Pr[X=x,Y=y]=\sum_y Pr[X=x\vert Y=y]\cdot Pr[Y=y]$

But I'm having a hard time understanding this intuitively. I can't quite seem to visualize this...

Now, I understand the law of total probability allows us to find a probability given a sample space is partitioned into pairwise disjoint sets and then some event exists "on top" of that sample space and we can sum the conditional probabilities of that event occurring given an event in the sample space has occurred. But how does this work with discrete random variables? Don't both discrete random variables partition the sample space?

Would I be correct to envision the law of total probability as it relates to discrete random variables in this way? (see below)

enter image description here

For example, say I have two discrete random variables $X$ and $Y$ and $X=\{8,9,10\}$ and $Y=\{1,2,3,4,5,6\}$. Assume that $X$ takes on the values of a biased three-sided die with probability distribution

$Pr(X=8)=\frac{1}{5}, Pr(X=9)=\frac{3}{5}, Pr(X=10)=\frac{1}{5}.$

And that $Y$ takes on the values of a fair six-sided die which has a uniform probability distribution. Then how does the law of total probability apply in this case?

Here is what I imagine an explicit use of the formula would look like:

$\sum_y Pr[X=x\vert Y=y]\cdot Pr[Y=y]=(Pr[X=x\vert Y=1]\cdot \frac{1}{6})+(Pr[X=x\vert Y=2]\cdot \frac{1}{6})+(Pr[X=x\vert Y=3]\cdot \frac{1}{6})+(Pr[X=x\vert Y=4]\cdot \frac{1}{6})+(Pr[X=x\vert Y=5]\cdot \frac{1}{6})+(Pr[X=x\vert Y=6]\cdot \frac{1}{6}).$

But in this case would we then need to have $X=x$ ranging across all values of $x$? I'm having a hard time envisioning this in an intuitive way. The above image is inuitive to me but I can't relate this to a specific case such as what I mention above that involves discrete random variables.

  • $\begingroup$ Some familiarity with contingency tables (very fundamental and useful) should help. $\endgroup$ – leonbloy Nov 18 '16 at 22:15
  • $\begingroup$ Are your example $X$ and $Y$ independent or dependent? $\endgroup$ – Henry Nov 19 '16 at 0:16
  • $\begingroup$ @Henry the examples are independent. But I was under the impression that this law of total probability applies to both independent and dependent discrete random variables since it is used in proving linearity of expectation, that is, the property that $E[X+Y]=E[X]+E[Y]$ for both independent and dependent discrete variables. $\endgroup$ – ClownInTheMoon Nov 19 '16 at 0:20
  • 2
    $\begingroup$ If they are independent then $\Pr[X=x \mid Y=y] = \Pr[X=x]$ for all $y$ so your righthand side becomes $6 \cdot \Pr[X=x] \cdot \frac16$ which is indeed $\Pr[X=x]$ as predicted $\endgroup$ – Henry Nov 19 '16 at 0:23

Let us say that your random experience is to throw two 6-side fair die. Let $X$ be the value of the first die and $Y$ be the value of the second die.

Consider the event $A =$ the total outcome is greater than $9$

We now want to calculate $P(A)$. Consider the partition of $A$ into $\{A_{10}, A_{11}, A_{12}\}$ where $A_{i}$ is the event where the total outcome is $i$.

What we want now is $P(A)$. But if that happens, it either happened because $A_{10}$ happened, because $A_{11}$ happened or because $A_{12}$ happened. Thus

$$P(A) = P(A_{12}) + P(A_{11}) + P(A_{10})$$

How to calculate each $P(A_i)$? Let us start with $P(A_{10})$.

We know that $X \geq 4$ because the dice can only roll up to 12, since the largest value one can attain is only $6$. Thus $P(A_{10})$ depends on the probabilities $P(Y \geq 4 | X = 6), P(Y \geq 5 | X = 5), P(Y \geq 6 | X = 4)$. In what way though? Well, the law of total probability tells us:

$$P(A_{10}) = P(Y \geq 4 | X = 6)\cdot P(X = 6) + P(Y \geq 5 | X = 5)\cdot P(X = 5) + P(Y \geq 6 | X = 4)\cdot P(X = 4)$$

With similar trains of thought we see that

$$P(A_{11}) = P(Y \geq 5 | X = 6)\cdot P(X = 6) + P(Y \geq 6 | X = 5)\cdot P(X = 5)$$


$$P(A_{12}) = P(Y \geq 6 | X = 6)\cdot P(X = 6)$$

Which we can then add up together to find $P(A)$.

Is it now clearer to you?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.