Consider the following formula, which is from the Binomial distribution: $$P(X=k)=\binom{n}{k}p^k\big(1-p\big)^{ n-k}$$

On the left hand side, it is the probability of one event: $\{\omega\in\Omega:X(\omega)=k\}$. One the right hand side, one can consider $n$ independent variables, and using the addition and multiplication formula. It seems that the underlying sample spaces can be different for the two sides of the equality. For the left hand side, as I understand, the sample space(up to set isomorphism) $\Omega_1=\{(a_i)_{i=1}^{n}:a_i=0,1\}$. And the random variable $X:\Omega_1\to{\mathbb R}$, $$X(\omega)=\sum_{i=1}^{n}a_i$$ While the right hand side, it can be $\Omega_2=\{0,1\}$. And for the random variable $Y:\Omega_2\to{\mathbb R}$, $Y(0)=0,Y(1)=1$.

Here is my question:

How can one reach the equality by working with different sample spaces(and hence different probability spaces)?


According to the comments and the answers to the question, I would like to modify the question as the following:

Is it possible to interpret the formula $P(X=k)=\binom{n}{k}p^k\big(1-p\big)^{ n-k}$ in terms of the following probability space $(\Omega,\Sigma,P)$ where

  1. $\Omega=\{0,1\}$;
  2. $\Sigma=2^{\Omega}$;
  3. $P(0)=p,P(1)=1-p\quad 0\leq p\leq 1$.
  • $\begingroup$ The sample space is $\{ 0, 1 \}^n$ for both sides. What's the confusion? $\endgroup$ – Qiaochu Yuan Jun 28 '11 at 16:45
  • $\begingroup$ $X$ is a measurable function from $\{ 0, 1 \}^n$ to $\mathbb{R}$. It sends a tuple to the number of $1$s in it. $\endgroup$ – Qiaochu Yuan Jun 28 '11 at 17:43
  • $\begingroup$ @Jack: How can the right-hand side correspond to a sample space with only two elements? $\endgroup$ – Shai Covo Jun 28 '11 at 17:43
  • $\begingroup$ @Shai Covo: If one regards the results of the n trials as n independent events, one can get the right hand side. $\endgroup$ – Jack Jun 28 '11 at 19:47
  • 1
    $\begingroup$ @Jack: But "$n$ independent events" does not correspond to a sample space with only two elements... $\endgroup$ – Shai Covo Jun 28 '11 at 19:51

On the right-hand side, all I can see is a number. That number could be found in several ways.

There is no random variable mentioned on the right-hand side of the equation, and no sample space.

And in any case, one can (and often does) show that two probabilities are equal when the underlying sample spaces are entirely different. Think of tossing a coin, event "Head", and tossing a die, event "even number."

  • $\begingroup$ Why was this downvoted? I'm curious… $\endgroup$ – ShreevatsaR Jun 28 '11 at 19:30
  • $\begingroup$ @ShreevatsaR: Me too! But comfortable with it, I want to get down to a multiple of $5$. If the answer was incorrect, that would be a different matter. $\endgroup$ – André Nicolas Jun 28 '11 at 19:37
  • $\begingroup$ Don't worry, you are not alone; Shai was downvoted too (and not by me)! $\endgroup$ – Emre Jun 28 '11 at 19:52
  • $\begingroup$ @Emre: Have not downvoted any answer to anything yet. On random variables, some people don't like the message that choice of sample space, random variable is not handed down from above. But I was on the nasty side in writing (correctly) that I did not see a random variable on the right. $\endgroup$ – André Nicolas Jun 28 '11 at 21:05

Let $(\Omega,\mathcal{F},P)$ be any probability space, and let $X:\Omega\to\mathbb{R}$ be any random variable (i.e. any measurable function). Then, by definition, we say that $X$ has the binomial distribution if there exists $p\in[0,1]$ and $n\in\{0,1,2\ldots\}$ such that $$ P(\{\omega:X(\omega)=k\}) = \binom nk p^k(1-p)^k, $$ for all $k\in\{0,1,2,\ldots,n\}$.

As an example, we could fix some $p\in[0,1]$ and $n\in\{0,1,2\ldots\}$, and then take $\Omega=\{(a_i)_{i=1}^n:a_i=0,1\}$, take $\mathcal{F}$ to be the power set of $\Omega$, take $P$ to be the probability measure on $(\Omega,\mathcal{F})$ determined by the identity $$ P(\{a_1,\ldots,a_n\}) = \prod_{i=1}^n p^{a_i}(1 - p)^{1 - a_i}, $$ and finally take $X$ to be the random variable defined by $$ X((a_i)_{i=1}^n) = a_1 + \ldots + a_n. $$ We would then have to go through the calculations and prove that $X$ has the binomial distribution.

But this is just one example. Here is another. Let $p\in[0,1]$ and $n\in\{0,1,2\ldots\}$ be given. Let $\Omega=\mathbb{R}^n$, let $\mathcal{F}$ by the collection of Lebesgue measurable sets, and let $P$ be the standard Gaussian measure defined by $$ P(A) = \frac1{(2\pi)^{n/2}}\int_A \exp\left({-\frac12(x_1^2+\cdots+x_n^2)}\right)\,dx_1\cdots dx_n. $$ Choose $x_0\in[-\infty,\infty]$ such that $$ \frac1{\sqrt{2\pi}}\int_{-\infty}^{x_0}e^{-x^2/2}\,dx = p. $$ Define $H:\mathbb{R}\to\mathbb{R}$ by $H(x)=1$ if $x\le x_0$, and $H(x)=0$ otherwise. Finally, define $X:\Omega\to\mathbb{R}$ by $$ X(x_1,\ldots,x_n) = \sum_{i=1}^n H(x_i). $$ It might be a little harder to check in this case, but one can still go through the calculations and verify that $X$ has the binomial distribution. (In this case, the components of $(x_1,\ldots,x_2)$ are independent, standard Gaussians on $\mathbb{R}$, and $X$ is the number of them which are less than or equal to $x_0$.)

Finally, one last example. Let $\Omega=\{0,1,2,\ldots,n\}$, let $\mathcal{F}$ be the power set, and let $P$ be determined by the identities $$ P(\{k\}) = \binom nk p^k(1-p)^k, $$ for all $k\in\Omega$. It now follows that if $X(\omega)=\omega$, then $X$ has the binomial distribution.

The point here is that the distribution of a random variable does not tell us what the underlying probability space is, or what the function $X$ is. Different random variables on different probability spaces can have the same distribution. The distribution of $X$ is just a rule telling us how to determine $P(X\in A)$ for all measurable sets $A\subset\mathbb{R}$. It does not tell us what $X(\omega)$ is, nor does it tell us what $\Omega$ is.


Seeing the edited question, the answer is (in general) no. Any random variable $X$ on $(\Omega,\Sigma,P)$ can take on at most two values. So if $p\in(0,1)$ and $n\ge 2$, then $X$ cannot have the binomial distribution. However, the probability space $(\Omega^n,\Sigma^n,P^n)$ is exactly the probability space I used above in the first example.

To add further details, there are only four {0,1}-valued random variables on $(\Omega,\Sigma,P)$. They are $$ U(0) = U(1) = 0;$$ $$X(0)=1, X(1) = 0;$$ $$Y(0)=0, Y(1)=1;$$ $$Z(0)=Z(1)=1. $$ Of these four, only $X$ is Bernoulli with parameter $p$. There are no others. So it is not possible to construct $n$ independent Bernoulli($p$)'s on $(\Omega,\Sigma,P)$. To do that, one must use $(\Omega^n,\Sigma^n,P^n)$.


Your idea concerning the right-hand side is not clear (and must be wrong). Anyway, one may find useful the following derivation of the equality $P(X=k) = {n \choose k} p^k (1-p)^{n-k}$. Let $$ \Omega = \{ (a_1 , \ldots ,a_n ):a_i \in \{ 0,1\} ,i = 1, \ldots ,n\} $$ and, for each $k=0,1,\ldots,n$, define $$ A_k = \bigg \{ (a_1 , \ldots ,a_n ) \in \Omega :\sum\limits_{i = 1}^n {a_i } = k \bigg \} . $$ Note that $A_k$ contains ${n \choose k}$ elements. Since, for any $\omega \in A_k$, $$ P(\{ \omega \}) = p^k (1-p)^{n-k}, $$ it thus follows that $$ P(A_k) = {n \choose k} p^k (1-p)^{n-k}. $$ (Note that $\sum\nolimits_{k = 0}^n {P(A_k )} = \sum\nolimits_{k = 0}^n {{n \choose k} p^k (1 - p)^{n - k} } = (p + (1 - p))^n = 1$.) Now, if we define $X: \Omega \to \mathbb{R}$ by $$ X(\omega ) = \sum\limits_{i = 1}^n {a_i } $$ (where $\omega = (a_1,\ldots,a_n)$), then $$ P(X=k) = P(\{ \omega \in \Omega :X(\omega ) = k\} ) = P \bigg(\bigg \{ (a_1 , \ldots ,a_n ) \in \Omega :\sum\limits_{i = 1}^n {a_i } = k \bigg \}\bigg) = P(A_k ). $$ Hence the equality $$ P(X=k) = {n \choose k} p^k (1-p)^{n-k}. $$

EDIT (in view of OP's Edit and comment above): The random variable $X$ can be decomposed as $$ X = X_1 + \cdots + X_n, $$ where the $X_i$ are independent random variables with $P(X_i = 1)=p$ and $P(X_i = 0)=1-p$, $i=1,\ldots,n$. While each $X_i$ can take only two values ($0$ or $1$), the $X_i$ (and their sum $X$) are defined on the same probability space. Specifically, $X_i : \Omega \to \mathbb{R}$ is defined by $X_i (\omega ) = a_i$, where $\omega=(a_1,\ldots,a_n)$. (Note that $X(\omega ) = \sum\nolimits_{i = 1}^n {X_i (\omega )} = \sum\nolimits_{i = 1}^n {a_i } $.)

  • $\begingroup$ Thank you very much for your work. I will reconsider about this issue. $\endgroup$ – Jack Jun 28 '11 at 20:43
  • $\begingroup$ @Jack: Thanks. Indeed, there is a lot to consider here, in particular in view of user11867's answer. $\endgroup$ – Shai Covo Jun 28 '11 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.