# Intuition behind independence & conditional probability

This is a basic question.

I have a good intuition that $A$ is independent of $B$ if $P(A \vert B) = P(A)$, and see how you can easily derive from this that it must hold that $P(A,B) = P(A)P(B)$.

But the first statement is not normally taken as a definition; instead the second is.

What is the intuition, or even derivation behind defining $A$ and $B$ as independent iff $P(AB) = P(A)(B)$?

The kind of explanation I am looking for would be one similar to that given by Jaynes for the definition of conditional probability in the first chapter of Probability: The Logic of Science, or even a Kolmogorov axiomatic explanation would help.

-
The two are equivalent by Bayes' theorem. –  Qiaochu Yuan Oct 4 '12 at 5:27
I know they are equivalent, please read the question –  zenna Oct 4 '12 at 5:30
For us to use your suggestion, one would need a separate definition of $\Pr(X|Y)$. –  André Nicolas Oct 4 '12 at 5:32
I think you've got a great "motivated" answer by @IttayWeiss. He presents a plausible story of how one might have proceeded and arrived at the current definitions. Are you satisfied with his answer or are you still looking for a better one? –  Assad Ebrahim May 19 '14 at 8:27

Arguing from the intuitive idea of probability (be it frequentist, Bayesian, or a la Janes), what can we say about $P(AB)$? Let us assume that $P(A)\le P(B)$. Since $AB\subseteq B$ we can safely deduce that $P(AB)\le P(B)$. By looking at well known and elementary examples it is easy to be convinced that $P(AB)$ can attain any value between $0$ and $P(B)$. But examining these cases shows that extreme values, close to $0$ or close to $P(B)$ are obtained when information about $B$ having occurred either severely conflicts with $A$ occurring (to get close to $0$), or strongly correlates with $A$ occurring (to get close to $P(B)$).

Now, more mathematically, one value in the range of $P(AB)$ that appears naturally is, of course, $P(A)P(B)$, so it is natural to investigate when that would occur. Notice that this value is symmetric in $A$ and $B$. Since the exact location of $P(AB)$ in its possible range seems to be highly sensitive to whether, and how, $A$ and $B$ influence each other we must conclude that the special value $P(A)P(B)$, being symmetric in the arguments, means that the mutual influences are neutral. That neutrality is another way of thinking about independence. Thus, we turn the intuition into a definition and say that $P(AB)=P(A)P(B)$ holds if, and only if, $A$ and $B$ are independent.

-
+1 -- this is a "motivated" definition, in the sense of the OP's question, i.e. it presents a plausible story of how one might have proceeded and arrived at the current definitions. Very nice! –  Assad Ebrahim May 19 '14 at 8:15

They are equivalent when $\: P(B) \neq 0 \:$.

The problem with $\: P(A|B) = P(A) \:$ is figuring out what $P(A|B)$ would mean if $\: P(B) = 0 \:$.

-

As a definition, $P(A,B) = P(A)P(B)$ uses intuitively simple concepts involving the probability both events happen and the probabilities each of then happens. It may not be intuitive for everyone why this is the definition, but it is intuitive what it is.

$P(A \vert B) = P(A)$ uses the less intuitively simple concept of conditional probability, which needs both definition and understanding.

-