Background info: I've taken a measure theory course, and I've read up on the basics of measure-theoretic probability theory. I'm trying to understand how to rigorously define iid random variables in a practical example.

Suppose we conduct an experiment to estimate the expected value of a random variable $X$ which measures the height of a population. To frame this all rigorously, let's say our probability space $(\Omega, \mathcal{F}, P)$ consists of a finite number of people, together with the uniform distribution, e.g. for $\omega_1, \omega_2 \in \Omega$, we have $P(\omega_1)=P(\omega_2)$. Define $X(\omega)$ to be the height of person $\omega$. In our experiment, at random we pick $n$ people from the population and measure their height, representing each individual person's height with the random variable $X_i$.

The Law of Large Numbers says if $X_i \sim X$ and the $X_i$ are iid, then the sample average tends to $E[X]$. My question, though, is how do I define the $X_i$? Each $X_i$ should represent a different person's height-- but if each $X_i$ is an indicator variable, then the $X_i$ are not the same distribution as $X$. It might make sense to set $X_i=X$, since then the $X_i$ are identically distributed, but then the $X_i$ are not independent.

I've seen posts that prove the existence of iid sequences in probability spaces, usually involving some infinite product of spaces, but I want an example of iid variables in a concrete setting. I'm currently studying some probability theory, and I find it easiest to understand the abstract definitions by relating them to intuitive examples.

  • $\begingroup$ I did not understand your statement "if each $X_i$ is an indicator variable, then the $X_i$ are not the same distribution as $X$." This problem seems to have nothing to do with indicator variables. Of course, you will have trouble finding an infinite iid sequence from a finite population. What if you just imagine an infinite sequence of coin flips? $\endgroup$ – Michael Aug 21 '16 at 7:07
  • $\begingroup$ The $X_i$ are not indicator random variables, they are samples drawn from the population. $\endgroup$ – Graham Kemp Aug 21 '16 at 7:09
  • $\begingroup$ But how do you define a sample drawn from the population? To invoke the LLN, It has to be another random variable with the same distribution as $X$. But other random variables on our sample space have the same distribution as $X$? $\endgroup$ – Uthsav Chitra Aug 21 '16 at 7:12
  • $\begingroup$ Michael: I guess for the LLN you want an infinite iid sequence, so sure, we could talk about coin flips instead. I know it doesn't have to do with indicator variables, but it was one of my ideas for defining the $X_i$. $\endgroup$ – Uthsav Chitra Aug 21 '16 at 7:15
  • $\begingroup$ Well, what is wrong with considering an infinite sequence of coin flips, each flip equally likely to be heads (0) or tails (1) and independent of the vector of all previous flips? $\endgroup$ – Michael Aug 21 '16 at 8:15

Here's a concrete example of iid random variables. Let $X$ be a random variable uniformly distributed on the interval $(0,1)$. Let $X_i$ be the $i$th digit in the (non-terminating) binary expansion of $X$. Clearly $P[X_i=0]=P[X_i=1]=1/2$. Moreover, $X_1,X_2,X_3,\ldots$ are mututally independent.

  • $\begingroup$ I like that example! I think it's the same as the LLN framework Did mentioned above/I've seen before to prove iid variables exist, but I like how it's initially framed in a concrete way. $\endgroup$ – Uthsav Chitra Aug 23 '16 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.