why measure theory for probability?

Question

I studied elementary probability theory. For that, density functions were enough. What is a practical necessity to develop measure theory? What is a problem that cannot be solved using elementary density functions?

Answer 1

The standard answer is that measure theory is a more natural framework to work in. After all, in probability theory you are concerned with assigning probabilities to events (sets)... so you are dealing with functions whose inputs are sets and whose outputs are real numbers. This leads to sigma-algebras and measure theory if you want to do rigorous analysis.

But for the more practically-minded, here are two examples where I find measure theory to be more natural than elementary probability theory:

1. Suppose $X \sim \text{Uniform}(0,1)$ and $Y = \cos(X)$. What does the joint density of $(X,Y)$ look like? What is the probability that $(X,Y)$ lies in some set $A$? This can be handled with delta functions but personally I find measure theory to be more natural.

2. Suppose you want to talk about choosing a random continuous function (element of $C(0,1)$ say). To define how you make this random choice, you would like to give a p.d.f., but what would that look like? (The technical issue here is that this space of continuous functions is infinite-dimensional and so Lebesgue measure cannot be defined). This problem is very natural in the field of Stochastic Processes including Financial Mathematics -- a stock price can be thought of as a random function. Under the measure theory framework, you talk in terms of probability measures instead of p.d.f.'s, and so infinite dimensions do not pose an obstacle.

Answer 2

I come from a probability and statistics background, so I kinda get where the OP is coming from. The basic question is about the motivation behind learning measure theory when working with probability. And sure enough, when I look at many measure theory texts, the motivation for learning measure theory--Lebesgue measure in particular--is that certain pathological sets don't work well within the framework of Jordan measure or the Riemann integral. The motivation that Stein and Shakarchi, as well as Tao give, include sets like the Cantor set, or cosets of irrational numbers, as the reasons why we need a more robust theory for integration.

This motivation about pathological sets makes sense if you are coming from an analysis background, because so much effort is put into defining number systems, and functions on those number systems. So the Cantor set is an important set within the context of Analysis. However, I have never seen anyone try to assign probability to the Cantor set in any practical application. I am not saying that someone has not already done this, it is more likely that this problem was solved long ago--though I have not checked.

I think the more sensible motivation for why to learn measure theory as a probabilist or statistician has to do with finding a way to measure the probability of infinite sets of events. In probability and statistics we need a way to identify the probability that a continuous random variable takes a value less than or equal to a specific value, for example the probability density function $$ pdf(X) := \Pr(X \leq x_0) $$ where $x_0$ is some particular value within the support of $X$. For example, say we want $pdf(X) = \Pr(X \leq 0.5)$. Given that $X$ is continuous, the set of all possible values of $X$--also known as the support of $X$--is the set $\{x : x \in [0,1], x \in \mathbb{R} \}$. The set of all possible values in the event of interest: namely $ X \leq 0.5$ is the set $\{x : x \in [0,0.5], x \in \mathbb{R}\}$.

Both the support of $X$ and the event of interest are infinite sets. Intuitively, the size or cardinality of the support should be larger than the cardinality of the event of interest--though lacking a theory of measure we have no way to make that notion precise. But think if you were to try and figure out the frequency or probability of this event. You would have to divide $\infty \div \infty$.

So measure gives us a way to assign probability to sets of event where each individual event has zero probability. Another way of saying this is that measure theory gives us a way to define the expectations and pdfs for continuous random variables. Of course, most of this theory is usually towards the end of a book on measure theory textbook. But when you get to the parts about Fatou's lemma and Dominated Convergence, etc., then the applications to probability become more apparent.

From my own experience I actually rather liked learning measure theory. It really is not that difficult once you understand the intent. It just took a second to shift my focus from a probability frame of reference to an analysis frame of reference, but the arguments are generally pretty intuitive. Good luck.

Answer 3

Simple answer: Tossing a coin.

Longer answer: You know that you treat discrete events like the above with probability mass functions or similar, but continuous things with probability density functions. Imagine you had $X$ which is randomly uniform on $[0,1]$ half the time and $5$ the rest of the time. Perfectly reasonable thing, could easily come up. Doesn't fit into either framework.

Measure theory provides a consistent language and mathematical framework unifying these ideas, and indeed much more general objects in stochastic theory. It removes any necessity to distinguish between fundamentally similar objects, and crystallizes the relevant points out, allowing much deeper understanding of the theory.

Answer 4

Measure theory goes beyond probability theory. It generalizes our notion of length, area and volume.