# Thought-experiment for ordered/unordered sampling, with replacement

I am currently studying the textbook Statistical Inference by Casella and Berger. In a section on combinatorics, the authors state the following:

Example 1.2.19 (Sampling with replacement) Consider sampling $$r = 2$$ items from $$n = 3$$ items, with replacement. The outcomes in the ordered and unordered sample spaces are these.

The probabilities come from considering the nine outcomes in the ordered sample space to be equally likely. This corresponds to the common interpretation of "sampling with replacement"; namely, one of the three items is chosen, each with probability 1/3. It is seen that the six outcomes in the unordered sample space are not equally likely under this kind of sampling. The formula for the number of outcomes in the unordered sample space is useful for enumerating the outcomes, but ordered outcomes must be counted to correctly calculate probabilities.

Some authors argue that it is appropriate to assign equal probabilities to the unordered outcomes when "randomly distributing $$r$$ indistinguishable balls into $$n$$ distinguishable urns." That is, an urn is chosen at random and a ball placed in it, and this is repeated $$r$$ times. The order in which the balls are placed is not recorded so, in the end, an outcome such as $$\{1, 3\}$$ means one ball is in urn 1 and one ball is in urn 3.

But here is the problem with this interpretation. Suppose two people observe this process, and Observer 1 records the order in which the balls are placed but Observer 2 does not. Observer 1 will assign probability $$2/9$$ to the even $$\{ 1, 3 \}$$. Observer 2, who is observing exactly the same process, should also assign probability $$2/9$$ to this event. But if the six unordered outcomes are written on identical pieces of paper and one is randomly chosen to determine the placement of the balls, then the unordered outcomes each have probability $$1/6$$. So Observer 2 will assign probability $$1/6$$ to the event $$\{ 1, 3 \}$$.

The confusion arises because the phrase "with replacement" will typically be interpreted with the sequential kind of sampling we described above, leading to assigning a probability $$2/9$$ to the event $$\{ 1, 3 \}$$. This is the correct way to proceed, as probabilities should be determined by the sampling m mechanism, not whether the balls are distinguishable or indistinguishable.

Everything was fine until I read this part:

But if the six unordered outcomes are written on identical pieces of paper and one is randomly chosen to determine the placement of the balls, then the unordered outcomes each have probability $$1/6$$. So Observer 2 will assign probability $$1/6$$ to the event $$\{ 1, 3 \}$$.

I not sure whether I'm interpreting this correctly. Are the authors saying that, since there are six pieces of paper, and since one is chosen randomly to determine the placement of the balls, regardless of the probabilities transcribed onto said pieces of paper, we therefore have that there is $$1/6$$ probability for each of the outcomes? I ask because this reasoning seems absurd to me, so I'm not sure whether I'm misinterpreting it or it just is absurd.

I would greatly appreciate it if people would please take the time to clarify this.

• Perhaps you're over-thinking this...? There are six pieces of paper. Assuming a uniform draw, you get any piece with prob $1/6$. Each piece of paper has only one thing written on it: one of the six unordered sets. There are no probabilities or any other numbers or values "transcribed" on the paper. – antkam Jan 8 at 21:29
• @antkam Yes, this is what I was seeking clarification on -- to see if I was misinterpreting what was being said. The way you describe it seems reasonable. But, interpreting it as you have done, it just seems so obvious that I don't understand what the point of this section was? What is the point that the author is trying to make here? – The Pointer Jan 8 at 21:39
• It seems (my guess) this author X is saying other authors Y got it wrong when they (i.e. Y) unjustifiably declare that "with replacement" automatically means every unordered set is equally likely. In the opinion of X, "with replacement" does not mean that. In the opinion of X, equi-probable unordered sets can be only the case if the problem statement explicitly mentions there is uniform draw among the unordered sets (the six pieces of paper). I personally agree with X, and it seems you do too. :) – antkam Jan 8 at 21:45
• @antkam ok, this puts my mind at ease. Thank you for taking the time to clarify this. :) If you would like to post this as an answer, I will accept it so that we can close this question. – The Pointer Jan 8 at 21:48

## 1 Answer

It seems (my guess) this author X is saying other authors Y got it wrong when they (i.e. Y) unjustifiably declare that "with replacement" automatically means every unordered set is equally likely. In the opinion of X, "with replacement" does not mean that. In the opinion of X, equi-probable unordered sets can only be the model if the problem statement explicitly mentions there is uniform draw among the unordered sets (the six pieces of paper). I personally agree with X, and it seems you do too. :)

I.e. X was "preaching to the choir" - but in such a way as to confuse the choir. :P