# Infinite number of samples are possible?

I came across this in an introductory stats book:

The idea of sampling error comes into play with the recognition that an infinite number of samples are possible. You could take one sample, then another, then another. You could continue the process time and time again.

Caldwell, Sally (2012-07-24). Statistics Unplugged (Page 101). Cengage Textbook. Kindle Edition.

I don't understand how this is possible, unless repeat samples are allowed.

Suppose I have a population of size 10000. Wouldn't the number of unique possible samples simply be 10000 choose 1 + 10000 choose 2 + ... + 10000 choose 9999?

• You can take samples with replacement. It seems that this is allowed in this statement. – callculus Jan 27 '17 at 10:27
• You mean replacement at the population level? So as I'm picking samples, the population changes, allowing an infinite number of possible samples? – user2313267 Jan 27 '17 at 10:29
• In the mathematical idealization the sampling being (and the world) might live forever, taking infinite many samples. – mvw Jan 27 '17 at 10:33
• You have a population size of n and you take a sample size of m. After you have picked a sample the population size is n-m. You put back the sample. The population size is n again. You can repeat this process infinitely times. – callculus Jan 27 '17 at 10:36
• Think of a dice. In principle, you can throw the dice infinite many often, although only $6$ different events are possible. – Peter Jan 27 '17 at 10:46

Also if you make measurement of the heights of a large number of people from a particular population, you may conclude that about 95% of them have heights between 60 and 75 inches. You might write this as $P(60 \le X \le 75) = .95,$ where $X$ represents the height of a randomly chosen person. You might measure heights to the nearest inch or to the nearest tenth of an inch, but either way you will get a lot of tied heights in a very large sample. And in a huge population it would hardly matter for practical purposes if someone happened to get measured twice. (If you could measure to any desired degree of accuracy, you would never get exactly the same result twice, but that would hardly be of practical value in understanding average heights of people or what fraction of the people are more than six feet tall.)
If you have a small population, then it is important to decide whether you are allowed to select a given element of the population more than once. There are somewhat different probability models depending on whether sampling is with replacement or without replacement. For many purposes, the difference isn't important unless the sample size $n$ is more than 10% of the population size $N.$