# Why does probability theory insist on sample spaces?

When you open a textbook on probability theory it'll routinely start by setting up some 3-tuple $(\Omega, \mathcal{A}, \mathbb{P})$ and then defining random variables as functions of type $\Omega \rightarrow \mathbb{R}$. But unless you're actually contemplating the throwing of dice and coins you'll never actually bother with the sample space in practice. Instead you'll simply say "let $X \sim \mathcal{N}(0,1)$...". If a sample space is ever required, it'll just be reverse engineered to fit the desired distribution function. So then why all this fuss about sample spaces? Why not just define your measure on the real numbers straight up? Is it just to make the dice throwers happy?

• It's mostly because not all subsets have a probability. So you want to be explicit and upfront about those that do. E.g. also here – Henno Brandsma Nov 22 '17 at 3:39
• The book excerpt in this answer should address your question. – Fabio Somenzi Nov 22 '17 at 3:40
• A very accessible and clear explanation of non-measurable sets is in this video. – Fabio Somenzi Nov 22 '17 at 3:49
• That's indeed a very common choice, but not all outcomes are real numbers. – Fabio Somenzi Nov 22 '17 at 4:54
• But yes, all these exercises starting with "Find a probability space Omega to model such and such random experiment..." are simply a waste of time, especially considering that they help convince their readers of the wrong notion that there is no choice in Omega. One die? Then $\{1,2,\ldots,6\}$, they say. Two dice? Then $\{1,2,\ldots,6\}^2$. And so on. Well, no, not necessarily... – Did Nov 22 '17 at 7:38

## 1 Answer

There is something beautiful about how probability theory treats general sample spaces $\Omega$. We have a lot of freedom to define the sample space any way that we want.

Ideally, we would like probability theory to handle any event that might be of interest. We would ordinarily describe an event with natural language such as "there are 3 blue cars in the parking lot this morning," or "Serena wins the tennis tournament." It is not obvious how such events can be mathematically treated. Remarkably, probability theory teaches us how to precisely handle events such as this! We first learn to be precise about our sample space $\Omega$. We then learn to view an event simply as a subset of $\Omega$. This idea is powerful. It allows rigorous probability results to be derived from simple and intuitive set theory principles of unions, intersections, and complements.

Probability theory allows us to use abstract sample spaces such as $$\Omega = \{H, T\}$$ or $$\Omega = \{\mbox{Wednesday, Thursday, Friday}\}$$ There are also cases where it is difficult to embed $\Omega$ into a subset of $\mathbb{R}$ or $\mathbb{R}^n$. For example, imagine choosing a random number $G \in \{1, 2, 3, ...\}$ and then generating $G$ additional variables $X_1, ..., X_G$, where each $X_i$ takes values in the set $\{1, ..., 100\}$. Outcomes for this system are most naturally represented by variable-length vectors $$(G, X_1, ..., X_G) \quad, G \in \{1, 2, 3, ...\}, X_i \in \{1, ..., 100\} \forall i \in \{1, ..., G\}$$ Notice that the length of the vector depends on the first component of the vector. This does not directly fit into $\mathbb{R}^n$ for any fixed $n$, yet probability theory easily handles this situation.

I agree that, when working with a finite number of random variables $(X_1, ..., X_n)$, it may be easiest to view the sample space as $\Omega = \mathbb{R}^n$. However, it should be emphasized that sample spaces that involve uncountably many real numbers are actually a lot harder to deal with than finite or countably infinite sample spaces. The most intuitive notions of probability come from studying finite or countably infinite $\Omega$. Conditional probabilities can be defined very easily in these cases. You can also assume that an "event" is any subset of $\Omega$, without dealing with "sigma-algebra" issues and "non-measurable set" issues. Things get more complicated when the sample space is uncountable. For example, it is not at all obvious how ot resolve the "divide by zero" issue when conditioning on the value $X$ of a Gaussian random variable.

So, a reasonable way to present probability theory is to present the general results that hold for general samples spaces $\Omega$ (defining the axioms and so on), present the standard intuition-building examples (coin tosses, card dealings, dice rolls, and so on), define conditional probability theory, the law of total probability, and the law of total expectation on finite or countably infinite sample spaces, and then define random variables and vectors that can (in some cases) have uncountable sample spaces. Hopefully, once you are comfortable working with general sets, you will be comfortable working with sets of real numbers. At this point, probability courses often focus on computational aspects of computing integrals, working with CDFs and PDFs, and converting the distribution of $X$ into a distribution for $Y$. These things are important, but it is also useful to remember the beautiful set theory foundations for probability theory that we learned at the start of the course!