# How to understand independence of probability?

By definition, when $$P(E\,|\,F) = P(E)$$ holds, we say that $E$ is independent of $F$.

By definition of conditional probability, $$P(E\,|\,F) = {P(E \cap F) \over P(F)} \Rightarrow P(E \cap F) = P(E)P(F).$$

I'm confused here that shouldn't $P(E \cap F) = 0$ if they are independent?

• You're getting confused with $E$ and $F$ being mutually exclusive, which does imply $P(E\cap F)=0$, but which is an entirely different concept than $E$ and $F$ being independent. – Zev Chonoles Jul 5 '15 at 6:38
• So that they are different concept! Thanks! – PHPIsTheBestLanguage Jul 5 '15 at 6:41
• If $P(E \cap F) = 0$, then $E$ and $F$ are definitely dependent, because if $E$ happens then $F$ cannot, and vice-versa. – Marconius Jul 5 '15 at 12:34
• Some years ago I reviewed the 2nd edition of a book from a reputable publisher. A social scientist, admittedly not a mathematician, insisted at the start that 'disjointness' and 'independence' are the same thing. As a result, many amazing and obviously untrue 'theorems' were 'proved.' Confusion on this point is evidently not limited to students beginning a probability class. – BruceET Jul 5 '15 at 16:47

Intuitively, saying that two logical propositions $E$ and $F$ are independent means that learning whether $E$ is true or false tells you nothing about $F$, and vice versa.

For example, suppose you flip two coins, a penny and a nickel, and consider the propositions

• $E$ = "The penny came up heads"
• $F$ = "The nickel came up heads"

For ordinary coins, it's very reasonable to assume that $E$ and $F$ are independent, because learning whether or not the penny came up heads should tell you essentially nothing about whether the nickel did. If you come up with a probability model where $E$ and $F$ are not independent, your model is saying something very strange about how these coins behave.

There are four possible outcomes for the coin flips, which can be listed in a truth table, using $1$ for "true" and $0$ for "false":

$$\begin{array}{r|rrr} E & 1 & 1 & 0 & 0 \\ F & 1 & 0 & 1 & 0 \end{array}$$

A typical statistical model for a pair of coin flips is to assume that each of the four possible outcomes is equally likely. In other words, \begin{align*} P(E \cap F) & = \tfrac{1}{4} & P(E \cap \neg F) & = \tfrac{1}{4} & P(\neg E \cap F) & = \tfrac{1}{4} & P(\neg E \cap \neg F) & = \tfrac{1}{4}. \end{align*} In this model, are $E$ and $F$ independent?

It's not hard to calculate from the probabilities above that $P(E)$ and $P(F)$ are both $\tfrac{1}{2}$, and $P(E \mid F)$ is also $\tfrac{1}{2}$. Hence, $E$ and $F$ are independent. This is good, because we argued earlier that $E$ and $F$ should be independent in any reasonable model for coin flips.

Since $E$ and $F$ are independent, the calculation in your question tells us that $P(E) P(F)$ should be equal to $P(E \cap F)$. Indeed, $\tfrac{1}{2} \cdot \tfrac{1}{2} = \tfrac{1}{4}$. Your condition $P(E) P(F) = P(E \cap F)$ is actually equivalent to independence: if the probabilities of two logical propositions satisfy your condition, the propositions are independent.

The two concepts of "independent" and "mutually exclusive" are different.

Two events are "mutually exclusive" ( that is, $$P(E \cap F) = 0$$) if they can't both happen at the same time. For example, if I roll a die and define E = "I roll a 6" and F = "I roll a 3", these can't both be true. If E happens, i know F didn't.

Two events are "independent" (that is, $$P(E \cap F) = P(E)P(F)$$) if the outcome of each has no influence at all on the other. For example if we each roll a die and define E = "I roll a 6" and F = "you roll a 3". Whether E happens or not, it makes no difference to the probability of F happening.

The only way a pair of events can be both "mutually exclusive" and "independent" is if at least one of them has zero probability.

(Note: as noted by wythagorus below, this is not the same as completely impossible - eg any exact result in a continuous distribution is possible but with zero prob)

• Small correction: Te only way a pair of events can be both "mutually exclusive" and "independent" is if at least oneof them has probability 0. This doesn't mean that it is impossible, it just means that it is extremely unlikely. (e.g. you throw a dart exactly at the center of a given circle) – wythagoras Jul 5 '15 at 10:42
• @wythagoras Agreed, thanks - will update. – IanF1 Jul 5 '15 at 10:43

You're correct that if $E$ and $F$ are independent, then $P(E \cap F)=P(E)P(F)$. If $E$ and $F$ are independent, then it means that the result of $E$ has no influence on the result of $F$ and vice versa. For example:

I toss a quarter and a dime. The result of the quarter toss has no influence on the dime toss.

However, if $P(E \cap F)=0$, then they are mutually exclusive.

This is for example: I toss one quarter.

• $E :$ It is a head.
• $F :$ it is a tail.

They cannot happen at the same time.