I know this is very basic. But very puzzling too and often missed by learners like myself.

When talking about continuous random variables with a particular probability distribution, what are the underlying sample spaces?

Second Q: why these sample spaces are omitted often times, and one simply says r.v. $X$ follows a uniform distribution on interval $[0,1]$? Isn't the sample sample space critically important? Many thanks!


You can take it to be a subset of $\mathbb{R}$ or, more generally, $\mathbb{R}^n$. A random variable uniformly distributed in $[0, 1]$ can be thought of as a random variable on the sample space $[0, 1]$ with probability density function $1$.

The sample space is in fact not critically important. (You may find it convenient to pick one to do computations in, but it doesn't matter which one you pick. This is analogous to choosing coordinates to do computations in linear algebra.) This point is explained very clearly in Terence Tao's notes here:

At a purely formal level, one could call probability theory the study of measure spaces with total measure one, but that would be like calling number theory the study of strings of digits which terminate. At a practical level, the opposite is true [emphasis added]: just as number theorists study concepts (e.g. primality) that have the same meaning in every numeral system that models the natural numbers, we shall see that probability theorists study concepts (e.g. independence) that have the same meaning in every measure space that models a family of events or random variables. And indeed, just as the natural numbers can be defined abstractly without reference to any numeral system (e.g. by the Peano axioms), core concepts of probability theory, such as random variables, can also be defined abstractly, without explicit mention of a measure space; we will return to this point when we discuss free probability later in this course.

Terence Tao's explanation of free probability is here. (I found it very enlightening; the framework Tao describes can be used to describe quantum probability with very little modification, unlike the measure theory framework.)

  • $\begingroup$ r.v.'s are measurable functions from sample space to R. So when we say a random variable $X$ uniformly distributed in $[0,1]$ means to me that the range of the r.v. $X$ is [0,1]. So you are saying we can think of the sample space for $X$ as any subset of $\mathbb R$, including the range of $X$ [0,1]? $\endgroup$ – Qiang Li Jan 19 '11 at 21:49
  • $\begingroup$ @Qiang Li: It would be $X: [0,1] \to \mathbb{R}$. $\endgroup$ – PrimeNumber Jan 19 '11 at 21:54
  • $\begingroup$ @Qiang: I'm not sure I understand the question. If you want to, you can think of the sample space for X as any interval [a, b] with the appropriate uniform probability measure; all of these measure spaces are isomorphic to [0, 1]. $\endgroup$ – Qiaochu Yuan Jan 19 '11 at 22:04

While I certainly cannot do better than Tao (and admit that I haven't read Qiaochu's links) I'd like to make the following point:

I see the reason for essentially restricting to $(\Omega,\Sigma, \mu) = [0,1]$ as a rather technical one. If one would like to have sequences of i.i.d. random variables, one needs to be able to form the product space $(\Omega,\mu)^{\mathbb{N}}$ - you need some form of Kolmogorov's consistency theorem. If you allow too nasty $\Omega$'s product spaces can be very degenerate. There are examples in Halmos's or Neveu's books where they show that a countable product can be carrying the trivial measure even if the factors themselves are far from trivial. The point is that $\Omega$ might simply be "too big" to be reasonable.

A very flexible and technically useful class of "reasonable" measurable spaces is the class of standard Borel spaces (for which Kolmogorov's consistency theorem holds, luckily). By definition these are the measurable spaces which are (measurably) isomorphic to the Borel $\sigma$-algebra of a complete and separable metric space. Here's the surprise (Hausdorff, von Neumann):

Every uncountable standard Borel space is isomorphic to $[0,1]$ with the Borel $\sigma$-algebra. Moreover, every non-atomic probability measure on a standard Borel space is equivalent to Lebesgue-measure on $[0,1]$.

So from this point of view there is essentially no restriction in assuming $\Omega$ to be $[0,1]$ to begin with. Of course, atoms are not really an issue. Since we deal with a probability space, there are at most countably many of them, so we'll just get a union of an interval and some countable set.

  • $\begingroup$ Note that I'm not saying that you should actually replace your $\Omega$ by $[0,1]$ just because you can, in principle. But if you're proving abstract results, this replacement is often technically convenient and powerful. $\endgroup$ – t.b. Jan 19 '11 at 23:26

Begin with a probability space $(\Omega,\mathcal{F},P)$ ($\mathcal{F}$ is called a $\sigma$-algebra on $\Omega$, $P$ is called a probability measure). The collection of all Borel sets of $\mathbb{R}$ is denoted by $\mathcal{B}(\mathbb{R})$. A mapping $X:\Omega \to \mathbb{R}$ is a (real-valued) random variable if it is $\mathcal{F}$-measurable, that is, $\{ \omega \in \Omega: X(\omega) \in B \}$ is in $\mathcal{F}$ for each $B \in \mathcal{B}(\mathbb{R})$. Write $P[\{ \omega \in \Omega: X(\omega) \in B \}]$ as $P[X \in B]$. As a mapping of $B$, this is a probability measure on $\mathcal{B}(\mathbb{R})$, which can be denoted as $P_X$ and called the distribution of $X$.

Now, to say that a random variable $X$ on $(\Omega,\mathcal{F},P)$ follows a uniform distribution on the interval $[0,1]$ simply means that $P_X$ is the measure on $\mathcal{B}(\mathbb{R})$ satisfying $P_X (\mathbb{R}-[0,1]) = 0$ and $P_X (I) = b-a$ for any interval $I \subset [0,1]$ with endpoints $a<b$. For the simplest example, take the probability space $(\Omega,\mathcal{F},P) = ([0,1],\mathcal{B}([0,1]),P)$, where $P$ is the restriction to $[0,1]$ of the measure just defined above, and define $X:\Omega \to \mathbb{R}$ by $X(\omega) = \omega$. Then, for any $B \in \mathcal{B}(\mathbb{R})$, $$ P_X (B) = P[X \in B] = P[\{ \omega \in [0,1]:X(\omega ) \in B\} ] = P[[0,1] \cap B], $$ from which it follows that $X$ is uniform on $[0,1]$. (Note that it is not essential that $\Omega$ be the set $[0,1]$.)


the sample space are the numbers that your random variable X can take. If you have a uniform distribution on the interval (0,1) then the value that your r.v. can take is any thing from O to 1. If your the r.v is outside this interval then your pdf is zero. So that means that the universal space are the all the real numbers but for this sample we are only consern with what happens from [0,1]


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.