# Samples and random variables

Suppose I picked a sample of $n$ 20-year-olds. I measured the height of each to obtain $n$ numbers: $h_1, h_2, \ldots, h_n$. According to theory of probability/statistics, there are $n$ random variables associated with the sample, say $X_1, X_2, \ldots, X_n$. However, I do not understand the relationship between the $X_i$ and the $h_i$, I have yet to see it explained clearly in any book and so I have a few questions.

1. What is the probability space corresponding to the $X_i$? It seems to me that the way one samples should effect what the probability space will look like. In this case, I am sampling without replacement and the order in which I picked the individuals is irrelevant so I believe the sample space $\Omega$ should consist of all $n$-tuples of 20-year-olds such that no two tuples contain the same individuals. In this way, $X_i(\omega)$ is the height of the $i$th individual in the $n$-tuple $\omega \in \Omega$. The sample I picked would therefore correspond to one particlar point in $\Omega$, call it $\omega_0$, such that $X_i(\omega_0) = h_i$. I surmise that the $\sigma$-algebra will be just the power set $2^\Omega$ of $\Omega$ but I haven't a clue as to what the probability measure would be.

2. Let $(\Gamma, 2^\Gamma, P)$ be a probability space where $\Gamma$ is the set of all 20-year-olds and let $X$ be a random variable on $\Gamma$ such that $X(\gamma)$ is the height of the individual $\gamma\in\Gamma$. What is the connection between the $X_i$ and $X$ besides that afforded to us by the law of large numbers? In particular, what is the exact relationship between the probability space of $X$ and that of the $X_i$?

-

Your experiment consists of choosing $n$ people from a certain population of 20-year-olds and measuring their heights. $X_i$ is the height of the $i$'th person chosen. In the particular outcome you obtained when you did this experiment, the value of $X_i$ was $h_i$.
The sample space $\Omega$ is all ordered $n$-tuples of distinct individuals from the population. Since the sample space is finite (there being only a finite number of 20-year-olds in the world), the $\sigma$-algebra is indeed $2^\Omega$. If you choose your sample "at random", the probability measure assigns equal probabilities to all $\omega \in \Omega$.

-
Suppose $n = 3$ and my sample consists of Alice, Bob and Eve. Are you saying that (Alice, Bob, Eve) and (Bob, Alice, Eve) would constitute two different points of the sample space? In my mind, it should not because I do not care about the order. – echoone Sep 9 '11 at 0:43
If you want $X_1$, $X_2$, $X_3$ to be random variables, you need to know which is which. So yes, the order does count in that case. If you only want to define random variables that are invariant under permutations, you can use a sample space of unordered $n$-tuples. – Robert Israel Sep 9 '11 at 1:27
I had always understood that $X$ is itself the random variable, and $X_i$, which is the height of the $i$th child, are the values assumed by $X$. – gary Sep 9 '11 at 4:33
Assuming the $X$ is the one I defined in my question, then I believe that the $X_i$ are defined such that $X_i(\omega) = X(\pi_i(\omega))$ where $\pi_i$ is the projection onto the $i$th coordinate. – echoone Sep 9 '11 at 18:10

When this kind of question arise, I cannot help thinking that somebody ought to mention the following:

To some extent, the purpose of probability theory is to avoid manipulating little-omegas.

Unfortunately, many first courses on the subject waste an incredible amount of time specifying what is Omega, which may leave the wrong impression that this is what the subject is really about. Simply put, this is nonsense, because all the quantities one needs are in fact "at the other end of the arrow".

The best option (and the only one which is used outside of first courses classrooms) is to establish once and for all that some suitable spaces Omega exist that are sufficient for the families of random variables one has in mind (and these do exist), and then to proceed happily. (Caveat: the structure of Omega may become an interesting question again, much later, but not before one begins to tackle questions of a much higher level of sophistication.)

Turning to the specifics of your question, one sees that the random variables $X_i$ and $X$ have the same distribution $\mu$. For a collection of $n$ individuals with heights $h_i$, $\mu$ is a discrete measure on $(\mathbb R,\mathcal B(\mathbb R))$ defined by $$\mu(B)=n^{-1}\,|\{i\,;\,h_i\in B\}|,$$ for every $B$ in $\mathcal B(\mathbb R)$. Rigorously speaking, since you pick individuals without replacement, the $X_i$s are not independent. For example, if $\mu(\{h\})=k/n$ with $k\ne0$, $$\mathrm P(X_1=X_2=h)=k(k-1)/n^2\ne k^2/n^2=\mathrm P(X_1=h)\mathrm P(X_2=h).$$ As this example shows, in practice one never uses the nature of $(\Omega,\mathcal F,\mathrm P)$ since all that matters is the fact that the random variables $X_i$ are defined on a same probability space and what their distribution is. Every actual computation will only involve the measure $\mu$ on $(\mathbb R,\mathcal B(\mathbb R))$ (hence $\mu$ is at the so-called "other end of the arrow" $\Omega\to\mathbb R$), and more generally the distribution $\mu_n$ of the whole sample $(X_1,X_2,\ldots,X_n)$, which is a measure on $(\mathbb R^n,\mathcal B(\mathbb R^n))$. Note that $\mu_n$ is also at the "other end of the arrow", only this time the arrow is $\Omega\to\mathbb R^n$, and that one never needs to know what $\mathrm P$ or $\Omega$ actually are.

-

After mulling this over for a while, I have an answer to my question:

1. As Robert said, my experiment consists of choosing $n$ 20-year-olds and measuring their height. The outcome of my experiment is an $n$-tuple of numbers $(h_1,h_2,\ldots,h_n)$. Now typically the sample space for an experiment is defined as the set of all possible outcomes. However, I think it makes more sense to define the sample space of an experiment as any measurable space that contains all outcomes and events of interest. In my experiment, $(\mathbf R^n, \mathcal B(\mathbf R^n))$ will do as the sample space. The $X_i$ then are measurable functions from $(\mathbf R^n, \mathcal B(\mathbf R^n))$ to $(\mathbf R, \mathcal B(\mathbf R))$ such that $X_i(h_1,h_2,\ldots,h_n)=h_i$. The probability measure $m\,$ on $\mathcal B(\mathbf R^n)$ that yields the distribution of heights of the population of 20-year-olds, i.e. the distribution of the range of $X_i$, is in general unknown.

2. So what is the relationship between $(\mathbf R^n, \mathcal B(\mathbf R^n),m)$ and $(\Gamma, 2^\Gamma, P\,)$ and $X_i$ and $X$? If $X_i$ and $X$ are identically distributed, then $m \circ X_i^{-1} = P \circ X^{-1}$. However, whether $X$ and $X_i$ are identically distributed depends on how my experiment was done and the size of the population. With simple random sampling with replacement on an infinite population, everything should be kosher. Otherwise, things are not so clear cut anymore.

-
If one draws individuals uniformly and without replacement, all the random variables $X_i$ and $X$ follow the same distribution, even for finite populations. – Did Dec 14 '11 at 9:10