I'm drawing one single sample from an uncountable sample space. I know the probability of sampling any given single point is zero.

Now, what if I draw samples again and again and again, to infinity? What is the probability for any single point to be eventually drawn (as time goes to infinity)?

My guess is that it is still zero, since I cannot touch all the points in an continuum even if you give me infinite time. At infinity, I will still have infinite points left to touch. But is this reasoning correct? Every now and then a math professor starts talking about subtleties of this kind, but where do I find related bibliography?

In this case, what difference does it make to sample with replacement (after sampling a point, that point is not removed from the sample space) or without replacement (the sampled point is removed from the sample space)?


  • $\begingroup$ What do you mean? The probability of the next sample shouldn't depend on the previous result. $\endgroup$ – user2345215 Dec 1 '14 at 21:48
  • $\begingroup$ Your question is well-phrased, and all of your conclusions are correct! Regarding your last question: in this case it makes no difference whether the sampling occurs with replacement or without replacement. $\endgroup$ – M. Wind Dec 2 '14 at 6:23

Firstly, there is not such thing as "sampling at random", not from a continuum and not from any set with more than one element. You must specify a probability distribution for the sampling, otherwise, you are just pretending that you are performing some operation, usually pretending that you are using a uniform atomic distribution, which for infinite sets does not exist.

So, you are choosing points from a continuum according to some fixed probability distribution. It is then not the case that the probability of choosing any particular point is $0$. It depends on your probability distribution. For instance, if you probability distributions picks $1$ and $2$ with probability $1/2$ each, then of course the chance of choosing $1$ is not $0$.

So, you are choosing points from a continuum according to some fixed probability distribution for which the mass of each point is $0$. You repeat the process countably many time, independently. For a fixed point, what is the probability of choosing it at some point? Answer: $0$. Is there a difference between sampling with or without repetition? Answer: Essentially no.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.