Let's say I have an experiment where I repeatedly toss a coin; each toss is independent. I would like to define a random variable $X: \omega \rightarrow R$. $X$ is the number of failures before the first success. I would like to visualize the sample space of this experiment, but I'm having trouble. Is the sample space an infinite set containing potentially infinitely long sequences?

  • $\begingroup$ One question I have is whether we consider the sequence with no heads a part of $\Omega$. $\endgroup$ – Neo M Hacker Mar 13 at 22:04

There are always infinitely many valid ways to choose the sample space. Here are four natural ones:

  1. The set of all infinite sequences of $T$ and $H$. The function $X$ gives the number of $T$s at the beginning of the sequence. In particular, $X(T,T,T,\dots)=\infty$.

  2. The set of all infinite sequences, except for the all tails sequence. Now, $X$ is a finite number for all inputs.

  3. The set of all finite sequences whose last entry is $H$ and whose other entries are all $T$. Here, we are taking examples $1$ or $2$ and ignoring some information.

  4. The set of nonnegative integers. $X$ is the identity function. The probability measure is $P(\{n\})=(1-p)^{n}p$, where $p$ is the probability of heads. This is the same as example $3$, with the correspondence $T^nH\longleftrightarrow n$.

It does not matter whether we include the all tails sequence because the probability of it is zero. You can remove any probability zero event from a sample space without changing anything.

  • $\begingroup$ thanks for the answer. "You can remove any probability zero event from a sample space without changing anything." This may be a stupid question. In example 1, won't the probability of each outcome (infinitely long sequence) be 0? If we remove all of them, won't we end up with an empty set? $\endgroup$ – Neo M Hacker Mar 13 at 23:24
  • 1
    $\begingroup$ @NeoMHacker You can remove any single probability zero event, but that does not mean you can remove all of them at the same time! In fact, it is OK to simultaneously remove countably infinitely many null events... but the space of all sequences is uncountable. $\endgroup$ – Mike Earnest Mar 13 at 23:27
  • $\begingroup$ Interesting. I don't have the background to understand why it's okay, but thanks for sharing. $\endgroup$ – Neo M Hacker Mar 13 at 23:32
  • 1
    $\begingroup$ Here is a simpler example to realize what is going on. Consider a random variable which is the roll of a die. There are two probability spaces which describe this. (1) A finite set $\{1,2,\dots,6\}$ where each element has probability $1/6$. $X$ is the identity function. (2) The continuous interval $[0,6)$, where the probability of any subset of the line is its legnth. $X$ is a piecewise constant function, where $X(\omega)=1$ for $\omega\in [0,1)$, $X(\omega)=2$ for $\omega\in [1,2)$, etc. $\endgroup$ – Mike Earnest Mar 13 at 23:48
  • 1
    $\begingroup$ There is no deeper explanation. Some random variables can be described by both infinite and finite probability spaces. Your comment about increasing the sample space and proportionally changing events works for finite set, but not for going from finite to infinite. $\endgroup$ – Mike Earnest Mar 13 at 23:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.