Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I was assisting once the course in Probability Theory where students learnt quite quickly that there are ways to assign the uniform distribution to any finite set - or even subsets of $\mathbb R$ of a finite measure. They also clearly understtod the proof that there are no ways to put a uniform distribution (as a probability measure) on $\mathbb Z$ or a uniform distribution in a sense of a constant density on $\mathbb R$. But then I was asked the question:

Why is it impossible? What is intuitive meaning of this?

The only answer I could find that time refers to $\mathbb R$: well, the density is just a density w.r.t. Lebesgue measure - so why do we want the uniform distribution to have a constant density w.r.t. precisely Lebesgue measure? (as for me, if we omit the algebraic structure on $\mathbb R$, it would be just a measure).

However, I couldn't answer the question why there is no a uniform distribution on $\mathbb Z$ where the sentence "each value comes with equal probability" does have sense, I hope. Now this question bothers me myself and any help is appreciated.

share|improve this question
    
"each value comes with equal probability"- what is the value of this probability? –  tards Dec 1 '11 at 17:07
    
@tards: even if it is $0$ - on $\mathbb R$ for any non-trivial continuous distribution you have $P(\xi = x) =0$. Let me stress the point that I am not asking about possible contradictions which does not allow uniform distribution to exist - I am asking, what is the intuitive reason? Infinitely many elements? $[0,1]$ also has infinitely many of them. Unboundness? You may introduce a metric on $\mathbb Z$ which makes it bounded. Compactness? Well, $(0,1)$ is not compact. –  S.D. Dec 1 '11 at 17:12
2  
Is it fair to rephrase your question as "why do we require $\sigma$-additivity for probabilities?" There are finitely additive probability measures that are translation invariant, both on $\mathbb{Z}$ and on $\mathbb{R}$ and I would argue that translation invariance is a way of expressing "uniformity". –  t.b. Dec 1 '11 at 17:39
1  
Here's a related question that I recently answered. –  Michael Hardy Dec 2 '11 at 0:13
5  
If each member of $\mathbb{Z}$ has the same probability $p>0$ and $\Pr(\mathbb{Z})=1$ then $p+p+p+\cdots=1$. Is there a reason why telling the students that if $p>0$ then $p+p+p+\cdots=\infty$ doesn't do the job? –  Michael Hardy Dec 2 '11 at 16:27

3 Answers 3

up vote 4 down vote accepted

If you have a uniform distribution $\mu$ on $\mathbb{R}$, it should assign the same value $v$ to $[z,z+1)$ for every $z\in\mathbb{Z}$. But if $v=0$, you have $\mu(\mathbb{R})=\sum_{z\in\mathbb{Z}}0=0$. On the other hand, if $v>0$ then $\mu(\mathbb{R})=\sum_{z\in\mathbb{Z}}v=\infty$. So $\mu(\mathbb{R})=1$ can not be when $\mu$ is countably additive, which is the usual assumption in probability. The same argument applies to the nonexistence of a uniform distribution on $\mathbb{Z}$, when you look at the probability assigned to each point.

Let's say that a uniform distribution on $\mathbb{Z}$ is a function $\mu:\mathcal{A}\to[0,1]$, with $\mathcal{A}$ being an algebra, such that, $\mu(\emptyset)=0$, $\mu(\mathbb{Z})$, $A,B\in\mathcal{A}$ and $A\cap B=\emptyset$ implies that $\mu(A\cup B)=\mu(A)+\mu(B)$ (so we require $\mu$ to be oly finitely additive). Lastly, the uniformity is expressed by the property that for all $A\in\mathcal{A}$, $A+z=\{a+z:a\in A\}\in\mathcal{A}$ and $\mu(A)=\mu(A+z)$.

Under these assumptions, a uniform distribution on $\mathbb{Z}$ exists. Let $\mathcal{A}$ consists of all subsets of $\mathbb{Z}$ that are finite or have a finite complement. This is an algebra. Let $\mu(A)=0$ if $A$ is finite and let $\mu(A)=1$ if $A$ has a finite complement. You can verify that this satisfies all requirements we made on a uniform distribution. It is even possible to generate such a uniform distribution with $\mathcal{A}=\mathcal{P}(\mathbb{Z})$, by using the Hahn-Banach theorem. You can find some details here, together with other notions of uniformity.

So the non-existence of a uniform distribution on $\mathbb{Z}$ is essentially a matter of accepting countable additivity. There is no "elementary" reason.

share|improve this answer

Let's look at the case of $\mathbb Z$ first. A natural way to define the uniformity of a probability distribution on a discrete set like $\mathbb Z$ is to say that a random variable $X \in \mathbb Z$ is uniformly distributed iff, for any subsets $A, B \subset \mathbb Z$,

$$\|A\| = \|B\| \iff \mathrm P(X \in A) = \mathrm P(X \in B),$$

where $\|S\|$ denotes the number of elements in the set $S$.

Now, let's make the further assumption that there exists some constant $M \in \mathbb Z$ such that $\mathrm P(|X| \le M) = \varepsilon > 0$. (A probability distribution that did not satisfy this assumption would have to be a very curious beast indeed, since a random variable so distributed would exceed any finite bound with probability $1$.) Let $A = \{-M,\, \dotsc,\, M\}$. Then we can decompose $\mathbb Z$ into countably many disjoint subsets $A_k = A + (2M+1)k$ such that $\|A_k\| = \|A\| = 2M+1$ for all $k \in \mathbb Z$, and thus $\mathrm P(X \in A_k) = \varepsilon$ for all $k \in \mathbb Z$. But then

$$\mathrm P(X \in \mathbb Z) = \sum_{k \in \mathbb Z}\, \mathrm P(X \in A_k) = \sum_{k \in \mathbb Z}\, \varepsilon = \infty.$$

Indeed, we can even take a finite union of sufficiently many sets $A_k$ to obtain a finite set $B$ such that $\mathrm P(X \in B) \ge 1$, which is clearly absurd for a probability distribution.

Similarly, a natural property one would require of a random variable $X$ uniformly distributed over $\mathbb R$ would be that, for all $a,b,c,d \in \mathbb R$, $a \le b$, $c \le d$,

$$b-a = d-c \iff \mathrm P(X \in [a,b]) = \mathrm P(X \in [c,d]).$$

We can then carry out a similar argument as above to show that, if there exists an $M \in \mathbb R$ such that $\mathrm P(|X| \le M) = \varepsilon > 0$, then there exists a bounded set $B \in \mathbb R$ such that $\mathrm P(X \in B) \ge 1$.

Of course, this is not a complete proof of the impossibility of a uniform probability distribution over $\mathbb R$ (or $\mathbb Z$), since I haven't actually proven that there might not be a strange probability distribution for which $\mathrm P(|X| \le M) = 0$ for all $M \in \mathbb R$, yet somehow $\mathrm P(X \in \mathbb R) = 1$. But hopefully I've at least convinced you that, if a uniform probability distribution over $\mathbb R$ or $\mathbb Z$ were to exist, it would have to seriously violate our intuition about how a proper distribution ought to behave.

share|improve this answer
    
@Didier: Thanks, noted. –  Ilmari Karonen Jan 27 '12 at 14:30

Think of it this way. You start dividing Z and R into N equal slices over a histogram. The overall probability has to sum to 1. So to know the probability to assign to each slice of the histogram you need to know the finite number of slices you are going to make. But Z and R are infinite in size. So you can't slice Z and R into any particular finite number of equal slices.

It is the equal size & equal probability of each slice that is the problem. Of course there are functions which have probability of 1 over infinity. For instance the normal distribution is defined for all values of X from -infinity to +infinity, and it sums to 1.


As an example consider dividing the set of integers into groups of 10. So if I have integers between 1 and 100, then I have 10 groups. Each group thus has a probability of 1/10 of being randomly chosen when I choose a random integer between 1 and 100.

But what happens if I extend set of integers to Z? I end up with an infinite number of groups and 1/infinity is undefined as far as probability goes. So I can't define a probability for a particular group of 10 being chosen.

share|improve this answer
2  
So you can't slice Z and R into any particular finite number of equal slices... Yes you can: to slice Z into two equal slices, consider 2Z and 2Z+1. –  Did Jan 27 '12 at 8:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.