# What does multiplication mean in probability theory?

For independent events, the probability of both occurring is the product of the probabilities of the individual events:

$Pr(A\; \text{and}\;B) = Pr(A \cap B)= Pr(A)\times Pr(B)$.

Example: if you flip a coin twice, the probability of heads both times is: $1/2 \times 1/2 =1/4.$

I don't understand why we multiply. I mean, I've memorized the operation by now, that we multiply for independent events; but why, I don't get it.

If I have $4$ bags with $3$ balls, then I have $3\times 4=12$ balls: This I understand. Multiplication is (the act of) scaling.

But what does scaling have to do with independent events? I don't understand why we scale one event by the other to calculate $Pr(A \cap B)$, if A, B are independent.

Explain it to me as if I'm really dense, because I am. Thanks.

• I really recommend reading This, and especially the section "Conditioning" which contains insight about independence of events. – user76568 Mar 30 '14 at 17:48
• Imagine a million people flip a coin twice. About half of them will get heads on the first toss. Of the people who got heads on the first toss, about half of them will get heads on the second toss. Thus, about $1/4$ of the people will get heads on both tosses. – littleO Mar 30 '14 at 18:47
• If there is a single blue ball, the probability of drawing it is one among $4\times3$, i.e. $1/12=1/4\times1/3$ (drawing the good bag among four and drawing the blue ball among three in this bag). – Yves Daoust Jan 7 '16 at 20:24
• @littleO I really like your example. Thank you! – user3019105 Feb 21 '18 at 17:12

I like this answer taken from http://mathforum.org/library/drmath/view/74065.html :

" It may be clearer to you if you think of probability as the fraction of the time that something will happen. If event A happens 1/2 of the time, and event B happens 1/3 of the time, and events A and B are independent, then event B will happen 1/3 of the times that event A happens, right? And to find 1/3 of 1/2, we multiply. The probability that events A and B both happen is 1/6.

Note also that adding two probabilities will give a larger number than either of them; but the probability that two events BOTH happen can't be greater than either of the individual events. So it would make no sense to add probabilities in this situation. "

• +1. This is such a beautiful way of explaining the crux of this rule. – Sathyam Dec 5 '16 at 19:49
• why did event B happen in 1/3 of no of outcomes of A? – Hydrous Caperilla Nov 12 '17 at 3:32
• I didn't understand this. Could you please elaborate? – Ardent Nov 6 '19 at 6:29

If you randomly pick one from $n$ objects, each object has the probability $\frac{1}{n}$ of being picked. Now imagine you pick randomly twice - one object from a set of $n$ objects, and a second object from a different set of $m$ objects. There are $n\cdot m$ possible pairts of objects, and thus the probability of each individual pair is $\frac{1}{n\cdot m} = \frac{1}{n}\cdot \frac{1}{m}$.

More generally, let $A$ be some event with probability $\Pr(A) = a$, and $B$ some other event with probability $\Pr(B) = b$. Assume you already know that $A$ happened, meaning that instead of looking at the whole probability space (i.e. at the whole set of possible outcomes), we're now looking at only $A$. What can we say about the probability that $B$ happens also, i.e. about the probability $\Pr(B\mid A)$ (to be read as "the probability of $B$ under the condition $A$")?

In general, not much! But, if $A$ and $B$ are independent, then by the definition of independence, knowing that $A$ has happened doesn't provide us with any information about $B$. In other words, knowing that $A$ has happened doesn't make the likelyhood of $B$ happening also any smaller or larger, so $$\Pr(B\mid A) = \Pr(B) \text{ if A,B are independent.}$$

Now look at $\Pr(A \cap B)$, i.e. the probability that both $A$ and $B$ happen. We know that if $A$ has happened, then $A \cap B$ happens with probability $\Pr(B\mid A)$. If we don't know that $A$ has happened, we have to scale this probability with the probability of $A$. Thus, $$\Pr(A \cap B)= \Pr(B\mid A)\Pr(A) \text{.}$$ [ You can imagine $A$ and $B$ to be some shapes, both inside some larger shape $\Omega$. $\Pr(A\cap B)$ is then the percentage of the area of $\Omega$ that is covered by both $A$ and $B$, $\Pr(A)$ the percentage of the area of $\Omega$ covered by $A$, and $\Pr(B\mid A)$ is the percentage of the area of $A$ covered by $B$. ]

If $A,B$ are independent, we can combine these two results to get $$\Pr(A\cap B) = \Pr(A)\Pr(B) \text{.}$$

• +1 for his nice explanation but how can compare two events if they are different from each other.For eg-getting a head in a coin and getting full marks in an exam.Are these independent events and if not ca u give some conditions for identifying independentt events – Hydrous Caperilla Nov 12 '17 at 6:00

You are still scaling, but by numbers that are smaller than $1$. In your example, you are scaling $1/2$ by a factor of $1/2$, scaling it down to $1/4$. The first $1/2$ represents the outcomes where the first coin flipped is heads. But only $1/2$ (the second "$1/2$" from your example) of those outcomes also have the second coin come up heads.

• Thanks+1 can you expand a little more? Multiplication is kind of confusing me. Since multiplication is a version of addition, can you explain in those terms? Thanks again – Emi Matro Apr 2 '14 at 1:15
• Multiplication by a whole number is repeated addition. But multiplication by something like $1/2$ is not really repeated addition, unless you are allowed to add a fractional amount of what you start with. But I don't see how that simplifies matters, since you are still left thinking about what half of that something is. Maybe your confusion is stemming from trying to always think of multiplication as repeated addition, when that's not the best thing to do. – alex.jordan Apr 2 '14 at 1:22
• I see. How should i think of multiplication if not repeated addition? Thanks – Emi Matro Apr 2 '14 at 1:45

Very informally, suppose that we flip the two coins, say a dime and a quarter, simultaneously $10000$ times. Then the number of times we get a head on the dime should be in the $5000$ range. If there is no "interaction" between the result on the dime and the result on the quarter, to get the approximate number of cases from these $5000$ in which we get a head on the quarter is obtained by scaling $5000$ by a factor of $\frac{1}{2}$.

Remark: Here is a fancier but less intuitive version. Let random variable $X$ be $1$ if the event $A$ occurs, and let $X=0$ otherwise. Define random variable $Y$ analogously. So our mean income if we get a dollar for each head on a dime is $\frac{1}{2}$, as is our mean income if we get a dollar for each head on a quarter. Now assume that the events $A$ and $B$ are independent, and we get a dollar only if both dime and quarter show a head. Then our average income from dimes alone gets scaled by a factor of $\frac{1}{2}$.

There are different ways to understand why multiplication is used; one stems from the origin of the multiplication rule, this explains why multiplication is used and where it comes from. I am afraid the "meaning in itself" of the multiplication isn't going to be illuminating because the multiplication here is derived from a definition (and definitions are assigned and somewhat arbitrary, except that a convention is established.) We could have said the probability of a certain event is π and one that will never occur is -π (the adjust the mathematics accordingly.)

What you are referring to is the multiplication rule of probability. This rule stems from the definition of an event occurring in basic probability. Namely; The probability that an event occurs is equal to the number of ways that it could possibly occur divided by the total number of outcomes. Keep this in mind because this simple idea is used to derive the multiplication rule of probability.

Probability of an event = Number of ways it can happen / total number of outcomes

Given two events (in this case event B happens after event A, and depends on the outcome of A... removing different colored marbles from a bag without replacement for example.)

The probability that event B happens given event A happened = the number of ways that b can happen when A happens / total number of ways that A happens.

Typically a Venn diagram is used to illustrate this (draw intersecting circles, label one of them A, the other one B, and the oval in the middle the "intersection of A and B" where intersection is denoted by a frowning symbol as follows "A ∩ B"

We can restate this in more formal terms as follows:

Pr(B∣A) = A ∩ B / Pr(A)

This is analgous to the definition of the probability of an event. Let me show you them side by side.

Pr(B∣A) = A ∩ B / Pr(A)

Probability of an event = Number of ways it can happen / total number of outcomes

The Probability of the event is Pr(B|A)

The number of ways it can happen is A ∩ B

The total number of outcomes is Pr(A)

This is simply an extension of the definition. Multiply both sides of this equation by Pr(A) and we arrive at a new equation:

A ∩ B = Pr(A)Pr(B|A) ... now you are wondering what is the meaning of "A ∩ B" in this context if it is supposed to refer to "Number of ways it can happen"...so look at the addition rule of probability. As follows:

P(A or B) = P(A) + P(B) - P(A ∩ B) ...Again this is more intuitive with Venn diagrams.

Rearranging the equation P(A ∩ B) = P(A) + P(B) - P(A or B)

https://math.stackexchange.com/a/2559825/531645

I had the same doubt. I think this is the best explanation based on counting.

Answer: Assume that we have an experiment with $M$ equiprobable outcomes($\{x1,x2,x3....\}$), $m$ of them considered favorable. Therefore, probability of success is $m\over M$ for first event. Now consider a second experiment with $N$ equiprobable outcomes($\{y1,y2,y3....\}$), $n$ of them are considered favorable. The probability of success of second event is $n\over N$.

When we assume two experiments are independent, we have total $M\dot N$ possible outcomes(All possible pairs = $\{ (x1,y1), (x1,y2)...,(x2,y1)\}$) in our sample space. Among these outcomes total favorable outcomes are $m\dot n$ (all possible pairs of m and n).

Therefore in case of independent experiments we multiply the probability.

$${mn\over MN}={m\over M}\cdot{n\over N}\ .$$

Please let me know if it helps to get an intuitive idea.

The question is old and an answer has already been accepted. However, I would like to express my opinion too, basically because I recently dived into probability too, and I hope that my answer helps someone and makes things clearer.

In its Udacity course "Intro to Statistics", Sebastian Thrun makes IMO a wonderful example using truth tables:

assume that $H$ = heads outcome after a single coin flip and that $T$ = tails outcome after a single coin flip. The probability $P(H\land H)$ (heads two times) is one of the following outcomes:

\begin{array}{|c|c|} \hline \mbox{1st flip} & \mbox{2nd flip}\\ \hline T & T \\ \hline T & H \\ \hline H & T \\ \hline H & H \\ \hline \end{array}

Therefore, we only have 1 case out of 4 which leads to the desired outcome $H \land H$ (heads two times). The probability $P(H \land H)$ is therefore $1/4$ .

But how does the table relate to the following formula $P(H \land H) = P(H) \times P(H)$ ?

If the probability $P(H) = 1/2 = 0.5 = 50\%$ , it means that for the first coin flip there are 2 out of 4 cases which could possibly satisfy $P(H \land H)$ (in blue):

$$\begin{array}{|c|c|} \hline \mbox{1st flip} & \mbox{2nd flip}\\ \hline T & T \\ \hline T & H \\ \hline \color{blue}{ H } & \color{blue}{ T } \\ \hline \color{blue}{ H } & \color{blue}{ H } \\ \hline \end{array}$$

Then, for the second coin flip, always given that $P(H) = 1/2 = 0.5 = 50\%$ , only a half of these 2 events would give us the desired outcome, and therefore only 1 event, only 1 row is the correct one (in red):

$$\begin{array}{|c|c|} \hline \mbox{1st flip} & \mbox{2nd flip}\\ \hline T & T \\ \hline T & H \\ \hline \color{blue}{ H } & \color{blue}{ T } \\ \hline \color{red}{ H } & \color{red}{ H } \\ \hline \end{array}$$

That said, probabilistically, on the second coin flip, only $1/2$ of the events (resulting in only 1 event in this case) of the half ($1/2$) of events of the first coin flip would lead us to the desired outcome $H \land H$ . Therefore:

$$P(H \land H) = P(H) \times P(H) = 0.5 \times 0.5 = 0.25$$

The half of the half of all events leads us to the desired event.

NOTE that for two independent events $X$ and $Y$ the formula $P(X \land Y) = P(X) \times P(Y)$ is also consistent with the specific case in which $P(Y) = 0$ ($Y$ could never happen).

In this specific case, we know that $P(X) = 1$ as it satisfies the equation $P(X) + P(Y) = 1$ (the sum of all the probabilities of each independent event is equal to 1).

What is the probability of $X \land Y$ ($P(X \land Y)$) then? If $Y$ could never happen, $X \land Y$ could never happen too, hence the probability is 0:

$$P(X \land Y) = P(X) \times P(Y) = 1 \times 0 = 0$$

TO SUM UP

If you think of probability of a particular variable to be equal to $X$ as the approximate fraction (portion) of $n$ events (each event of the $n$ events equals to only one variable) which could lead to the desired outcome $X$ and again if you think of probability of a particular variable to be equal to $Y$ as the approximate fraction of the same $n$ events which could lead to the desired outcome $Y$, you end up with $P(X) = 1/b$ and $P(Y) = 1/d$ with $b$ and $d$ being both $> 1$ .

You have $x = n \times P(X)$ which is the approximate number of events which can produce $X$ and $y = n \times P(Y)$ which is the approximate number of events which can produce $Y$, given $n$ events.

If you double the number of events, therefore you have $g = 2n$ events and you couple them starting from the first $2$ events:

$$\text{1st couple})\quad \text{Event 1}\quad \text{Event 2} \\ \text{2nd couple})\quad \text{Event 3}\quad \text{Event 4} \\ \text{3rd couple})\quad \text{Event 5}\quad \text{Event 6} \\ ... \\ \text{nth couple})\quad \text{Event g-1}\quad \text{Event g} \\$$

You end up with $n$ couples of 2 events and each couple can be seen as an event itself. Now you want to know what is the probability of a couple to lead to the variable $X$ for the first event of the couple and to $Y$ for the second event of the couple ($P(X \land Y)$).

You have $P(X) = 1/b$ of $n$ events ($n$ couples) which possibly led to $X$ on the first event of each couple, and on $1/b$ of $n$ events only $P(Y) = 1/d$ of the $1/b$ of $n$ events possibly led to $Y$ .

Hence $P(X \land Y) = P(X) \times P(Y)$

ANALOGY

Another way to think of the multiplication of probabilities is by using the analogy of permutations (though I would say that it is only a conceptual analogy and you should kinda take it with a grain of salt, but it helped me to get the idea):

If you have a string of 3 characters and each character differs from the others, how many permutations can you obtain from this string?

Let's say you have the following string:

$$abc$$

The possibile permutations are (including $abc$):

$$1)\quad abc \\ 2)\quad acb \\ 3)\quad bac \\ 4)\quad bca \\ 5)\quad cab \\ 6)\quad cba \\$$

The number of permutations equals $6$ and is equal to the factorial of $3$ (the $n$ number of characters of the word):

$$n! = 3! = 6$$

To prove it, you start with a string without characters. When you have to choose the first character, you have $3$ possibilities, hence you will at least have $3$ different permutations, each starting with one of the $3$ characters.

Once you have used your first $x$ character (it could be $a$ or $b$ or $c$), you have $2$ characters left ($b$ and $c$ if you picked $a$ initially). Therefore, for each of the $3$ permutations you have so far, you would at least have 2 additional permutations ($2$ permutations if you started with $a$, $2$ permutations if you started with $b$, $2$ permutations if you started with $c$).

So far, you have 2 permutations starting with $a$, 2 starting with $b$, and 2 starting with $c$.

You have in fact $6$ permutations ($3 \times 2$) and for each permutation you just add the last missing character which you didn't insert for each group. You don't create more permutations at this point, as you don't have more options, just the last missing character you didn't use yet for that specific permutation.

I use this analogy of permutations (2 permutations for each distinct character of the 3, $3 \times 2$) to think about the multiplication of probabilities. Think about the groups that formed during the analysis of truth table:

Starting from 4 events (the 4 rows of our truth table), we divided the events in 2 groups of 2 events, 1 group containing 2 events having the first desired outcome (obtain $H$ from the first coin flip). Of these 2 events, only a half of the 2 and therefore 1 event was the event which had again the desired outcome $H$.