What is the intuition for using sigma algebras to define probability spaces?

Question

When I was taught probability in school, we studied basic events like coin flips $\lbrace H,T\rbrace$ or rolling a die $\lbrace 1,2,3,4,5,6\rbrace$.

Let's take coin flips as an example. Suppose $\Omega =\lbrace H,T\rbrace$. $\mathcal{F} =\lbrace \emptyset,\lbrace H\rbrace,\lbrace T\rbrace, \lbrace H,T\rbrace \rbrace$.

Then

1. $\emptyset,\Omega \in \mathcal{F}$
2. Closed under complementation (e.g. if $A = \lbrace H\rbrace$ then $A^c = \lbrace T\rbrace$ and both are in $\mathcal{F}$)
3. Closed under countable unions (e.g. $A = \emptyset \cup \lbrace H\rbrace = \lbrace H\rbrace \in \mathcal{F}$)

So I get how it works in these simple examples.

But I don't get why all this fancy terminology is necessary. Why do I need this elaborate structure to be able to describe a coin flip? Isn't it enough just to have the event set $\Omega$ and a function $\Bbb P$ that can map each event onto a number? Why do I need the sigma algebra?

Answer 1

I'll try to give an intuitive explanation:

So the probability is defined as a measure, i.e. a map that maps subsets of $\Omega$ into values in $[0,1]$ that satisfies certain axiom we desire.

Naturally people would want to ask "can we find a good map that properly assigns a value to every subset of $\Omega$ that follows the axiom we desire?"

The answer is yes for the finite $\Omega$, but is complicated, and is usually a NO for infinite $\Omega$. (EDIT: to be more precise, yes for finite or countable space, and no for uncountable space).

Since we cannot properly assign values for all subsets of $\Omega$ in lots of cases, we start to ask "what is the collection of subsets that could be properly assigned values to, by our map/measure?". Preferably, we hope such collection to be as big as possible so that we could properly measure as much subsets as possible - usually we end up expanding such collection into a sigma-algebra.

EDIT - an example

Let $\Omega$ be the set of all points on the unit circle, and the action on $\Omega$ by a group $G$ consisting of all rational rotations (rotations by angles which are rational multiples of $\pi$). Thus $G$ is countable while $\Omega$ is uncountable. Hence $\Omega$ breaks up into uncountably many orbits under $G$, with each orbit that corresponds to rational number $q$ consisting of points that are $q\pi$ angle away from each other. Using the axiom of choice, we could pick a single point from each orbit, obtaining an uncountable subset $ A\subset \Omega$ with the property that all of its translates by $G$ are disjoint from $A$ and from each other - to see this: if otherwise, then $\exists a \in (A+q_1\pi) \cap (A+q_2\pi)$. But since there are no two elements in $A$ from the same orbits, we have $\arg((a-q_1\pi),(a-q_2\pi)) \notin \mathbb Q\pi$, contradiction. (here I use +/- to denote rotation of a point for certain angle)

The set of those translates partitions the circle into a countable collection of disjoint sets, which are all pairwise congruent (by rational rotations). Then the set $A$ will be non-measurable for any rotation-invariant countably additive probability measure on $\Omega$, because: if $A$ has zero measure, countable additivity would imply that the whole circle has zero measure. If $A$ has positive measure, countable additivity would show that the circle has infinite measure.

Answer 2

Let $S$ be any subset of the possible outcomes. Naively, one would like to think that it is meaningful to ask "What is the probability that the outcome is in $S$?"

It turns out that this naive belief runs^* into contradictions under very natural hypotheses. To fix this, we need to restrict which questions of the above form we're allowed to ask.

That is what the $\sigma$-algebra is — the collection of sets $S$ for which it is meaningful to ask the above question. The axioms of a $\sigma$-algebra reflect the sorts of arguments we think we can make for meaningful questions.

For example, if we can ask whether the outcome is in $S$, we ought also to be able to ask whether the outcome is not in $S$. Thus, if $S$ is in the $\sigma$-algebra, we insist $S^c$ is also.

(we could potentially be more conservative about what we believe is meaningful; in such a case we would modify our foundations by replacing the notion of $\sigma$-algebra with the appropriate structure)

---

When the sample space is finite (or countable), the naive belief is fine; we take the collection of all subsets of outcomes as the $\sigma$-algebra. We usually don't really pay any attention it, so it's easy to overlook that there was a choice involved here.

_{*: We can play set-theoretic tricks to eliminate the contradictions the naive belief leads to, but this forces us to reject things like the axiom of choice and the partition principle, which makes for a nasty set-theoretic universe}