5
$\begingroup$

Assume a probability triplet $(\Omega, \mathcal{F}, \mathbb{P})$.

My current understanding of $\mathcal{F}$ is that it must define events i.e. the subsets of $\Omega$ where probability is defined. I also understand that $\mathcal{F}$ must be closed under countable complements and unions. Is it correct to say that the intuitive reasoning is as follows:

  1. We need closure under complements by Kolmogorov's axioms. If you know $\mathbb{P}(A)$ then you know $\mathbb{P}(A^c) = 1 - \mathbb{P}(A)$.

What I don't understand is why we need closure under countable unions. Even, if we know $\mathbb{P}(A), \mathbb{P}(B)$, we cannot arrive at $\mathbb{P}(A \cup B)$ without knowing $\mathbb{P}(A \cap B)$. I have read this answer but the author of the answer neglects to answer why we need countable unions when the sets $A$ and $B$ are disjoint:

The last axiom is closed under countable unions. Let me give you another stupid example. Consider the roll of a die, or 𝑋={1,2,3,4,5,6}. What if I were to tell you the 𝜎 algebra for this is {∅,𝑋,{1},{2}}. That is, I know the probability of rolling a 1 or rolling a 2, but I don't know the probability of rolling a 1 or a 2. Again, you would justifiably call me an idiot (I hope the reason is clear). What happens when the sets are not disjoint, and what happens with uncountable unions is a little messier but I hope you can try to think of some examples.

Help?

$\endgroup$
  • 1
    $\begingroup$ $A\cap B =A^c \cup B^c$ which is in $\mathcal{F}$ if $A,B$ are in $\mathcal{F}$. $\endgroup$ – Nap D. Lover Jul 13 '19 at 16:43
  • $\begingroup$ @NapD.Lover : (+1) though minor mistake as you obviously meant $(A\cap B)^c$. $\endgroup$ – Michael Jul 14 '19 at 3:53
  • $\begingroup$ But you see, your reasoning might be circular. As soon as we close $\mathcal{F}$ under unions then we can calculate $A \cap B$. You cannot know the probability of a union of two sets if you do not know the probability of an intersection and vice versa. The theory needs us to be able to assign the probability of union of A and B independently of their independent probabilities. $\endgroup$ – Vykta Wakandigara Jul 14 '19 at 11:02
3
$\begingroup$

Why do the axioms need countable additivity?

Let $S$ be a (nonempty) sample space and $\mathcal{F}$ a sigma-algebra of events. A function $P:\mathcal{F}\rightarrow \mathbb{R}$ is a probability measure if it satisfies these standard axioms:

  • $P[A]\geq 0$ for all $A \in \mathcal{F}$.

  • $P[S]=1$.

  • (finite additivity) $P[A\cup B] = P[A]+P[B]$ whenever $A, B$ are disjoint sets in $\mathcal{F}$.

  • (countable additivity) If $\{A_i\}_{i=1}^{\infty}$ is a sequence of disjoint sets in $\mathcal{F}$ then $$ P[\cup_{i=1}^{\infty}A_i] = \sum_{i=1}^{\infty}P[A_i]$$

Why do we require the last bullet (countable additivity)? Real analysis gives us useful definitions for the sum of a finite or countably infinite number of nonnegative terms. Thus, we want our probability theory to allow both finite and countably infinite sums. This is why countable additivity is needed. With countable additivity we can say $$ \mbox{$\{A_i\}_{i=1}^{\infty}$ disjoint events} \implies P[\cup_{i=1}^{\infty} A_i] = \sum_{i=1}^{\infty} P[A_i]$$ In particular, if we partition the sample space $S$ into a countably infinite number of disjoint pieces, we want the sum of the probabilities of each piece to be 1. It is impossible to prove this from the first three bullets: The axioms must include the countable additivity axiom.

Indeed the following example is possible: Let the sample space be the natural numbers: $$S = \mathbb{N} = \{1, 2, 3, ...\}= \cup_{i=1}^{\infty} \{i\}$$ Let the sigma-algebra be the set of all subsets of natural numbers: $$\mathcal{F}=2^{\mathbb{N}}$$ We can build a function $P:2^{\mathbb{N}}\rightarrow\mathbb{R}$ that satisfies the first 3 bullets of the above axioms (but not the countable additivity bullet) such that $$ P[\mathbb{N}] =1 \neq 0 =\sum_{i=1}^{\infty} P[\{i\}] \quad (Eq. 1)$$ This is just weird. So we need the countable additivity axiom. How to build an example that satisfies (Eq. 1)? This is very difficult and uses a theory of Banach limits, the main idea is to define $P[A]$ for each $A \subseteq \mathbb{N}$ by: $$ P[A] = \lim_{n\rightarrow\infty} \frac{|A\cap \{1, 2, ...,n\}|}{n}$$ whenever the limit exists, and to define $P[A]$ using a Banach limit when the regular limit does not exist. It is easy to see that $P[A]=0$ for every finite set $A\subseteq \mathbb{N}$ and so the weird (Eq. 1) holds. It can also be shown that the first three axiom bullets hold for this example.

What if we change sigma-algebras to only use disjoint unions?

The probability axioms only require countable disjoint unions. Why not change the sigma-algebra definition to only require that the countable disjoint union of sets in the sigma-algebra are also in the sigma-algebra?

Let $S$ be a nonempty sample space. The standard definition of a sigma-algebra on $S$ is a collection $\mathcal{F}$ of subsets of $S$ such that (i) $S$ is in $\mathcal{F}$; (ii) if $A$ is in $\mathcal{F}$ then $A^c$ is in $\mathcal{F}$; (iii) If $A_1, A_2, ...$ is a countably infinite sequence of sets in $\mathcal{F}$ then $\cup_{i=1}^{\infty}A_i$ is in $\mathcal{F}$.

Wish-list for Alternative proposed definition: Let's say a "modified" sigma-algebra on $S$ is a collection $\mathcal{F}$ of subsets of $S$ such that:

  1. $S$ is in $\mathcal{F}$.

  2. If $A$ is in $\mathcal{F}$ then $A^c$ is in $\mathcal{F}$.

  3. If $A$ and $B$ are in $\mathcal{F}$ then $A \cup B$ is in $\mathcal{F}$.

  4. If $A_1, A_2, ...$ is a countably infinite sequence of disjoint subsets of $\mathcal{F}$ then $\cup_{i=1}^{\infty} A_i$ is in $\mathcal{F}$.

Well, how does this modified wish-list change the sigma algebra? Not at all. This set of 4 points on the wish-list is equivalent to the standard definition of sigma-algebra. This is because:

  • Induction can prove that, for every positive integer $n$, if $A_1, ..., A_n$ are in $\mathcal{F}$ then $\cup_{i=1}^n A_i$ is in $\mathcal{F}$.

  • The Nap D comment shows that if $A, B$ are in $\mathcal{F}$ then $A\cap B$ is in $\mathcal{F}$.

  • Any countably infinite union of sets in $\mathcal{F}$ can be written as a countably infinite disjoint union via $$ \cup_{i=1}^{\infty} A_i = \cup_{i=1}^{\infty} (A_i \cap (A_1\cup...\cup A_{i-1})^c)$$

Caveat: The sigma-algebra might change if we remove the part of our wish-list that says $A\cup B$ must be in $\mathcal{F}$. A counter-example and a discussion of Dynkin systems is here: Sigma-algebra requirement 3, closed under countable unions.


Note that $A \cup B$ is on our wish-list because with it we can prove $$ \boxed{P[A\cup B] = P[A] + P[B] - P[A\cap B]}$$ which is arguably one of the most intuitive (and important) results of probability theory. Probability theory would be overly restrictive if we did not have the above boxed equation.

$\endgroup$
  • $\begingroup$ Thank you, This is a very comprehensive answer. $\endgroup$ – Vykta Wakandigara Jul 14 '19 at 11:04
  • 1
    $\begingroup$ (+1) I would just like to add that countable additivity can be proven if we assume what some authors call “continuity from above” of the measure as an axiom instead. That is, if $A_n \supset A_{n+1}$ defines a decreasing sequence of nested sets such that $\cap_n A_n=\emptyset$ then $P(A_n)\to 0$. In fact Kolmogorov assumes this as his Axiom IV in chapter 2 of his Foundations of the Theory of Probability and then proves countable additivity on the next page as a theorem. $\endgroup$ – Nap D. Lover Jul 14 '19 at 17:24
  • 1
    $\begingroup$ Additional note: By induction we can prove (only using the first three axiom bullets, i.e., without countable additivity) that: If $\{A_i\}_{i=1}^{\infty}$ is a countably infinite sequence of disjoint sets in $\mathcal{F}$ then \begin{align} P[\cup_{i=1}^n A_i] &= \sum_{i=1}^n P[A_i] \quad \forall n \in \{1, 2,3, ...\}\\ P[\cup_{i=1}^{\infty}A_i] &\geq \sum_{i=1}^{\infty}P[A_i] \end{align} The inequality cannot be turned into equality without the countable additivity axiom (or some equivalent version of countable additivity). $\endgroup$ – Michael Jul 15 '19 at 5:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.