# Probability of picking a specific value from a countably infinite set

I have just learned in probability that picking a specific value from an uncountably infinite set (continuous) has a probability of zero, and that we thus estimate such things over an interval using integrals. This clearly does not apply to finite sets (discrete), where you can easily calculate the probability. But does it not apply to a countably infinite set (natural numbers for example), as it is discrete? On one hand, if we calculate limit of picking a certain element as one over x where x goes to infinity, it seems to be zero, but then again, it's discrete variable and I am not sure if it works the same way as continuous...

• It depends on the probability distribution you use. We can rule our the uniform distribution. – PrimeNumber Apr 3 '11 at 5:46
• @PEV: My question is, how is it different from continuous case? The uniform distribution does not work there either then, since probability of each single thing is 0? Could you elaborate? – user9034 Apr 3 '11 at 5:50
• – Shai Covo Apr 3 '11 at 6:44

I'm assuming that implicit in your question is that you're looking for a uniform distribution. (Otherwise, the statement "picking a specific value from an uncountably infinite set has a probability of zero" is false.)

To answer such questions systematically, you need a clear definition of what you mean by probabilities. You'll find the usual definition e.g. in the Wikipedia articles on probability axioms, probability measure and probability space. The key point there is that probabilities need to be countably additive. This allows you to derive a contradiction from assigning zero probability to elementary events in a countable probability space, but not in the case of an uncountable space. Assigning zero to a singleton set in a countable space leads to the contradiction that the countable sum of the zeros for all the singletons must be $0$ (from countable additivity), but $1$ because it's the probability for the entire space. Note that this has nothing to do with "discreteness" in a topological sense; e.g., it's true for the rationals, independent of whether you regard them as a discrete space or with the usual topology induced by the topology of the reals.

• thanks, that's what confused me, because I realize that for discrete variables we use summation, and summing zeros wont give you one... So there is no straightforward/easy way of determining a probability for uniform distribution across countably infinite set? I will read the articles you linked to right now. – user9034 Apr 3 '11 at 6:18
• @Probabiltiyq: Again, you first need a clear definition of "uniform probability". Any definition I can think of would imply that each elementary event has the same probability. In that case, not only is there no straightforward/easy way of determining a uniform distribution on a countably infinite set; there's no way at all. The same argument that shows that the uniform probability for an elementary event can't be zero also shows that it can't be finite, since otherwise these probabilities would all add up to $\infty$ -- so you can only get $0$ or $\infty$, but never $1$ as required. – joriki Apr 3 '11 at 6:34

When you compute the probability of choosing a particular value from a set (assuming that your axiom system allows this action) you do the following: assume all elements of the set are given an equal probability of being observed (the uniform distribution). Since the total of all probabilities must be $1$, each element is assigned a probability of $1/C$, where $C$ is the number of elements in the set; in this case $C$ is an infinite cardinal. Now whether it is a continuum cardinal or the countable cardinal doesn't really matter (modulo the answer given below). In both cases, the naive evaluation of the probability is zero. Of course, this does not preclude you from considering other scenarios ...

For instance, in the surreal number system, $\omega$ is the countable cardinal. In assigning a probability of $1/ \omega$ to each counting number (from $1$ to $\omega$) we can get a total sum of $1$, as required. But in the surreals, this probability value is not zero. It is $\epsilon$, an infinitesimal element added to the real numbers.

Just to extend Joriki's answer:

There is an additional issue that a sum of uncountably-many non-zero terms necessarily diverges (partition the terms of the sum by , e.g., sets Sn : Sn={x:x>1/n} . Then, by uncountability of the index, at least one of the Sn's will be infinite, say, for Sj={x:x>1/j}. Then the sum is bounded below by 1/j+1/j+.... (infinitely many times) -->oo . So there is no assignment (even a non-uniform one) to uncountably-many events in which more than countably-many will have non-zero probability, as the sum of the probabilities (i.e., the cumulative prob. function) would necessarily diverge. But you can have a countably-infinite collection whose elements add up to 1 (there are uncountably many, actually; e.g., take any series converging to N

Then you can assign the probabilities this way: Prob(X=e_i)=2^-i ; i=1,2,... Then your cumulative probability is 2^-1+2^-2+..... =1 You can of course do the same with any series b_n that adds up to one :

Prob (X=e_i)=b_i , i=1,2,...

As to uniform probabilities in an infinite set, consider the same argument as one used for uncountable: you will have infinitely many terms 1/n . Then this expression will be unbounded: add 2n-many to get two, 3n-many for 3,... kn-many for k (to bring-in heavy machinery, look up Archimedean principle). Edit: corrected my mistake pointed out by Joriki, from "more than uncountably many", to "more than countably many", in middle of top paragraph.

• I think you mean "in which more than countably many will have non-zero probability"? – joriki Apr 3 '11 at 6:51
• Yes, let me edit that. – gary Apr 3 '11 at 23:53