What is the intuition for using sigma algebras to define probability spaces?

When I was taught probability in school, we studied basic events like coin flips $\lbrace H,T\rbrace$ or rolling a die $\lbrace 1,2,3,4,5,6\rbrace$.

Let's take coin flips as an example. Suppose $\Omega =\lbrace H,T\rbrace$. $\mathcal{F} =\lbrace \emptyset,\lbrace H\rbrace,\lbrace T\rbrace, \lbrace H,T\rbrace \rbrace$.

Then

1. $\emptyset,\Omega \in \mathcal{F}$
2. Closed under complementation (e.g. if $A = \lbrace H\rbrace$ then $A^c = \lbrace T\rbrace$ and both are in $\mathcal{F}$)
3. Closed under countable unions (e.g. $A = \emptyset \cup \lbrace H\rbrace = \lbrace H\rbrace \in \mathcal{F}$)

So I get how it works in these simple examples.

But I don't get why all this fancy terminology is necessary. Why do I need this elaborate structure to be able to describe a coin flip? Isn't it enough just to have the event set $\Omega$ and a function $\Bbb P$ that can map each event onto a number? Why do I need the sigma algebra?

• One reason is that non-measurable sets can be an issue. That is, there's no sensible way to assign a probability to every subset of $[0,1]$ in proportion to its size, because there are some rather strange subsets that occur if you believe in the axiom of choice. Jun 23, 2017 at 21:38
• That issue doesn't arise for finite sets, though. I'm not sure why it's useful to think about $\sigma$-algebras in that case. Jun 23, 2017 at 21:39
• If you only intend to describe a coin flip, then no, you do not need all that. That's a rather modest objective, though, and people doing probability want to do much, much more. Jun 24, 2017 at 1:52
• You can exchange 'fancy' with 'paraphernalia'. Jun 24, 2017 at 2:59
• In addition to the accepted answer, the notion of conditional expectation and martingales use the notion of sigma algebra crucially, as a way to measure the "amount of information" available at some time in a process. Jun 24, 2017 at 6:05

I'll try to give an intuitive explanation:

So the probability is defined as a measure, i.e. a map that maps subsets of $$\Omega$$ into values in $$[0,1]$$ that satisfies certain axiom we desire.

Naturally people would want to ask "can we find a good map that properly assigns a value to every subset of $$\Omega$$ that follows the axiom we desire?"

The answer is yes for the finite $$\Omega$$, but is complicated, and is usually a NO for infinite $$\Omega$$. (EDIT: to be more precise, yes for finite or countable space, and no for uncountable space).

Since we cannot properly assign values for all subsets of $$\Omega$$ in lots of cases, we start to ask "what is the collection of subsets that could be properly assigned values to, by our map/measure?". Preferably, we hope such collection to be as big as possible so that we could properly measure as much subsets as possible - usually we end up expanding such collection into a sigma-algebra.

EDIT - an example

Let $$\Omega$$ be the set of all points on the unit circle, and the action on $$\Omega$$ by a group $$G$$ consisting of all rational rotations (rotations by angles which are rational multiples of $$\pi$$). Thus $$G$$ is countable while $$\Omega$$ is uncountable. Hence $$\Omega$$ breaks up into uncountably many orbits under $$G$$, with each orbit that corresponds to rational number $$q$$ consisting of points that are $$q\pi$$ angle away from each other. Using the axiom of choice, we could pick a single point from each orbit, obtaining an uncountable subset $$A\subset \Omega$$ with the property that all of its translates by $$G$$ are disjoint from $$A$$ and from each other - to see this: if otherwise, then $$\exists a \in (A+q_1\pi) \cap (A+q_2\pi)$$. But since there are no two elements in $$A$$ from the same orbits, we have $$\arg((a-q_1\pi),(a-q_2\pi)) \notin \mathbb Q\pi$$, contradiction. (here I use +/- to denote rotation of a point for certain angle)

The set of those translates partitions the circle into a countable collection of disjoint sets, which are all pairwise congruent (by rational rotations). Then the set $$A$$ will be non-measurable for any rotation-invariant countably additive probability measure on $$\Omega$$, because: if $$A$$ has zero measure, countable additivity would imply that the whole circle has zero measure. If $$A$$ has positive measure, countable additivity would show that the circle has infinite measure.

• interesting intuitive explanation (+1); however, if you could add a couple of examples ... (maybe the famous needle problem ?) Jun 23, 2017 at 21:57
• @GCab Thanks, and I'll try to look into some possible examples. Jun 23, 2017 at 23:20
• Why can't you assign subsets for some $\Omega$ and how does the sigma-algebra solve that problem? Jun 23, 2017 at 23:47
• Really terrific answer. This cleared up my question entirely. Thanks very much. Jun 24, 2017 at 3:16
• The issue is not really between sample spaces that are "finite" versus "infinite," rather it is between "finite or countably infinite" versus "uncountably infinite." In other words, countably infinite sample spaces are just as easy to work with as finite (they have uncountably many subsets, but we can define a measure for all of them via the mass function). Jun 24, 2017 at 8:41

Let $S$ be any subset of the possible outcomes. Naively, one would like to think that it is meaningful to ask "What is the probability that the outcome is in $S$?"

It turns out that this naive belief runs* into contradictions under very natural hypotheses. To fix this, we need to restrict which questions of the above form we're allowed to ask.

That is what the $\sigma$-algebra is — the collection of sets $S$ for which it is meaningful to ask the above question. The axioms of a $\sigma$-algebra reflect the sorts of arguments we think we can make for meaningful questions.

For example, if we can ask whether the outcome is in $S$, we ought also to be able to ask whether the outcome is not in $S$. Thus, if $S$ is in the $\sigma$-algebra, we insist $S^c$ is also.

(we could potentially be more conservative about what we believe is meaningful; in such a case we would modify our foundations by replacing the notion of $\sigma$-algebra with the appropriate structure)

When the sample space is finite (or countable), the naive belief is fine; we take the collection of all subsets of outcomes as the $\sigma$-algebra. We usually don't really pay any attention it, so it's easy to overlook that there was a choice involved here.

*: We can play set-theoretic tricks to eliminate the contradictions the naive belief leads to, but this forces us to reject things like the axiom of choice and the partition principle, which makes for a nasty set-theoretic universe