I'm a beginning student of Probability and Statistics and I've been reading the book Elementary Probability for Applications by Rick Durret.

In this book, he outlines the 4 Axioms of Probability.

  1. For any event $A$, $0 \leq P (A) \leq 1$.
  2. If $\Omega $ is the sample space then $P (\Omega) =1$.
  3. If $A$ and $B$ are disjoint, that is, the intersection $A \cap B = \emptyset$, then $$P(A\cup B) = P(A) + P(B)$$
  4. If $A_1, A_2,\ldots$, is an infinite sequence of pairwise disjoint events (that is, $A_i\cap A_j = \emptyset$ when $i \neq j $) then $$P\left(\bigcup_{i=1}^\infty A_i\right)=\sum_{i=1}^\infty P(A_i).$$

The book fails to explain why we need Axiom 4. I have tried searching on Wikipedia but I haven't had any luck. I don't understand how we can have a probability of disjoint infinite events. The book states that when the you have infinitely many events, the last argument breaks down and this is now a new assumption. But then the book states that we need this or else the theory of Probability becomes useless.

I was wondering if there were any intuitive examples of situations where this fourth axiom applies.

Why is it so important for probabilty theory? And why does the author state that not everyone believes we should use this axiom.

  • 7
    $\begingroup$ Possibly related: math.stackexchange.com/questions/564718/… $\endgroup$ – Brenton Dec 27 '15 at 1:57
  • $\begingroup$ de Finetti was a vocal critique of the countable additivity axiom (that is $(4)$). $\endgroup$ – A.S. Dec 27 '15 at 6:56
  • $\begingroup$ FYI 1 2 3 4 $\endgroup$ – BCLC Dec 27 '15 at 9:36
  • $\begingroup$ No, the question is why do you need axiom 3 when you have axiom 4? Just take $A_1 = A, \ A_2 = B, \ A_i = \emptyset \ \forall i \ge 2$. $\endgroup$ – Alex M. Jan 2 '16 at 20:00
  • $\begingroup$ @Brenton This is too advanced for a BEGINNER question. $\endgroup$ – Brother John Jan 3 '16 at 10:29

Notice that if $A,B,C$ are pairwise disjoint, you have \begin{align} \Pr(A\cup B\cup C) & = \Pr(A\cup B)+\Pr(C) & & (\text{since }A\cup B\text{ is disjoint from }C) \\[6pt] & = \Pr(A) + \Pr(B) + \Pr(C) & & (\text{since }A\text{ is disjoint from }B) \end{align} and the same can be done with any finite sequence of pairwise disjoint events just by applying Axiom 3 repeatedly (or to put it another way, by mathematical induction).

However, there are assignments of numbers to subsets of a set that satisfy the first three axioms while failing to satisfy the fourth. The fourth axiom rules out such systems. Here is a simple example: Suppose $A\subseteq \{1,2,3,\ldots\}$. Let $$ P(A) = \lim_{n\to\infty} \frac{|A \cap \{1,\ldots,n\}|} n. $$ For some sets (for example the set of all integers whose first digit is $5$) this limit fails to exist. However, for pairwise disjoint sets $A$ and $B$ for which the limit does exist, the third axiom holds, as do the first two. But the fourth axiom fails: $$ P\left( \bigcup_{n=1}^\infty \{n\} \right) = P(\{1,2,3,\ldots\}) = 1 \ne 0 = \sum_{n=1}^\infty 0 = \sum_{n=1}^\infty P(\{n\}). $$ Hence the fourth axiom is not redundant.

Now suppose you throw a coin repeatedly until the first time you get a "head". What is the probability that the number $X$ of trials is even? It is $$ \Pr(X\in\{2,4,6,8,10,\ldots\}) = \sum_{n\text{ even}} \Pr(X=n). $$ There you use Axiom 4.

\begin{align} & \Pr(X\in\{2,4,6,8,10,\ldots\}) = \sum_{n\text{ even}} \Pr(X=n) \\[6pt] = {} & \sum_{n\text{ even}} \Pr(\text{“tail'' on first }n-1\text{ trials and “head'' on } n \text{th trial}) \\[6pt] = {} & \sum_{n \text{ even}} \left( \frac 1 2 \right)^n = \frac{1/4}{1-1/4} = \frac 1 {4-1} = \frac 1 3. \end{align}

| cite | improve this answer | |
  • 3
    $\begingroup$ This has helped me tremendously. Thank you very much. I don't understand why the author didn't explain this concept and basically dismissed it and just gave it to us as is. $\endgroup$ – Brother John Dec 27 '15 at 2:51
  • $\begingroup$ @BrotherJohn : I glad it helped. ${}\qquad{}$ $\endgroup$ – Michael Hardy Dec 27 '15 at 18:50

Another thing to point is that the fourth axiom gives us a way to deal with countably infinite sequence of events. In another word, it allows us to take limit! $$P\left(\bigcup_{i=1}^\infty A_i\right)=\sum_{i=1}^\infty P(A_i)=\lim_{n\to \infty}\sum_{i=1}^n P(A_i).$$ While the first three axioms can only deal with finite events.

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.