To motivate my question, I start off with a very simple example of prediction problem.

Let's say Mike is interested in predicting the crime rate, which we denote as the random variable $y$, in the cities. And Mike identifies an array of available variables in play, which we denote as the random vector $\mathbf{x}$. Then we proceed by imposing various assumptions or using various approximation strategies to find a good approximation of the conditional expectation function $\mathbb{E}[y\mid\mathbf{x}]$.

It all seems to be a legitimate way to model the problem of predicting the crime rate in cities as a mathematical problem. However, one important definition is left unspecified, that is, what exactly is $y$ and $\mathbf{x}$. Sure, they are functions from the sample space $\Omega$ to the set of reals $\mathbb{R}$. But what is this $\Omega$? Most literature in statistical theory and econometrics theory does not attempt to explicitly specify them. The only times the sample space is specified is when we are doing toy examples like infinite coin tossing for learning purposes.

But of course, I understand in order to transform real world problems to mathematical models, we need to make assumptions. And in this case, we should in fact just assume the existence of a probability space where everything that any random variables that will ever come into play are measurable. And use real world data as realizations of these random variables to make inferences on certain relationships between random variables defined on this probability space.

However, not being able to get a glimpse into the sample space and working with functions whose domain we don't even know make me feel like I am not on a solid footing here. I have asked some people who seem to suggest in this case the $\Omega$ can be defined as the set of all possible states of a city. But how can one describe precisely the meaning of "possible states" here? How large is this set? I mean, we can't even describe an element of that set in precise language. There are also similar issues with defining our $\sigma$-algebra. Should we define it as the $\sigma-$algebra generated by all random variables? But then how many and what random variables should we even include as the generating set of random variables?

With all these in mind, what are some of the philosophical or mathematical reasons we assume such an appropriate probability space exist except mathematical convenience, and does the absence of a explicit probability space hinder the philosophical or mathematical soundness of empirical application of statistical theory based on measure theory? Finally, are there alternative theories of estimation and prediction that tries to address this issue of not being explicit (if this is an issue)?

  • $\begingroup$ Much of applicable statistical theory can be discussed without using the a measure-theoretic framework. Distributions of random variables can be specified in terms of PDFs, CDFs, and MGFs. Some derivations are (at least) much simpler and more elegant in a measure-theoretic setting. In my view, one example is the derivation of the Kolmogorov-Smirnov goodness-of-fit statistic from a Brownian Bridge. Perhaps others can extend the list of empirical results for which a measure-theoretic approach is especially helpful. $\endgroup$ – BruceET Jan 24 '18 at 19:27
  • 2
    $\begingroup$ To sum up: Not useful. Actually I fail to locate a single instance of application where knowing what Omega is, helps. For example, when throwing a dice, Omega the set of integers from 1 to 6 works; and Omega the interval [0,1] with the encoding of the result of the dice you are thinking about, works; and tons of other Omegas work; and this choice is irrelevant to the computations oneshall perform afterwards. $\endgroup$ – Did Jan 30 '18 at 10:19
  • $\begingroup$ To add to this, the first axiomatization of probability theory is not due to Kolmogorov but to John von Neumann and was created to axiomatize quantum mechanics. But one can show that it also encompasses the Kolmogorov axiomatization. The interesting part is that von Neumann starts with algebras of observables, i.e. the random variables and states on those variables, i.e. the probability measure. There is no underlying sample space needed. All you need is the structure of your algebras of observables and the state on them. $\endgroup$ – Raskolnikov Feb 1 '18 at 6:02
  • $\begingroup$ This paper might interest you. Since you are already familiar with the Kolmogorovian approach, you can skip in the text to the second chapter. $\endgroup$ – Raskolnikov Feb 1 '18 at 6:11
  • $\begingroup$ I also remember I answered a question about probability in quantum mechanics which explains a bit the von Neumann model of probability and how it connects with the Kolmogorov probability model. $\endgroup$ – Raskolnikov Feb 1 '18 at 6:34

There is really no question of philosophical or mathematical soundness. Probability theory is just as sound as any other sort of mathematical analysis. If you really wanted you could specify a suitable $\Omega$ for any particular application, but you wouldn't learn anything interesting. For soundness all you need to know is that some $\Omega$ will work.

Terry Tao's Foundations of probability theory is a good overview of the conventional foundations of probability theory as well as different approaches. There are a couple of phrases used to deal with $\Omega$ without actually creating too much unnecessary work and notation:

Let $\Omega$ be a sufficiently rich probability space.

This means we take $\Omega$ big enough to express any required random variable for the argument that is to follow. One way to implement this that would suffice for most purposes is to take $\Omega$ to be an infinite stream of independent uniformly distributed bits ($\{0,1\}$-valued random variables). Then, in the course of any particular argument whenever you need to consider a new random variable, you can assign some of the unused bits to encode your new random variable, making sure to leave infinitely many unused bits untouched for later use.

This is philosophically similar to the phrase "let $K$ be a sufficiently large integer", except that sometimes people care about explicit constants, whereas no-one ever cares about $\Omega.$

Extending $\Omega$ if necessary, ...

This is the approach emphasised most in Terry Tao's article. Whenever you need to consider a new random variable, you extend the sample space to model it. This means $\Omega$ is used for larger and larger sample spaces through the argument. If you were encoding this literally in a computer-checked proof you might use "variable shadowing" or give them separate names $\Omega_1,\Omega_2,\cdots$ (yuck!).

See Section 4 of Terry Tao's article for some alternate approaches to the foundations of probability, including measure algebras instead of measure spaces, and operator algebras (also mentioned by Raskolnikov in the comments). These don't really change the fact that we work with some ambient model like $\Omega$ that we want to be flexible about.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.