# What does a probability of $1$ mean?

From a textbook on probability on the Law of Large Numbers:

Theorem 3-19 (Law of Large Numbers): Let $X_1,X_2, \ldots , X_n$ be mutually independent random variables (discrete or continuous), each having finite mean and variance. Then if $S_n = X_1 + X_2 +\dots+ X_n$,

$$\lim_{n \to\infty} P\left(\left|\frac{S_n}{n} - \mu\right| \geq \varepsilon\right) = 0$$

Since $S_n$ is the arithmetic mean of $X_1,X_2, \ldots , X_n$, this theorem states that the probability of the arithmetic mean $\frac{S_n}{n}$ differing from its expected value $\mu$ by more than $\varepsilon$ approaches zero as $n \to \infty$. A stronger result, which we might expect to be true, is that $\lim_{n \to\infty} \frac{S_n}{n} = \mu$ but this is actually false. However, we can prove that $\lim_{n \to\infty} \frac{S_n}{n} = \mu$ with probability one.

The only difference between the last sentence and the one before that is the phrase 'with probability one'. What does probability one mean here ? The usual definition is that the event occurs with 100% certainty. If that is the case, why is the original assertion $\lim_{n \to\infty} \frac{S_n}{n} = \mu$ false ?

• If you want to understand how to use $\TeX$ well, take a look at my edits to your question. I changed $lim_{n->\infty}$ to $\lim_{n\to\infty}$ and $\displaystyle lim_{n->\infty}$ to $\displaystyle\lim_{n\to\infty}$ in several instances, and $(|\frac{S_n}{n}-\mu|)$ to $\displaystyle\left(\left|\frac{S_n}{n}-\mu\right|\right)$, and $X_1+X_2 . . . X_n$ to $X_1+X_2+\dots+X_n$, and changed various other things to standard $\TeX$ usage. May 24, 2013 at 13:44
• Essentially a duplicate of Zero probability and impossibility, only asking about $1$ instead of $0$.. May 24, 2013 at 16:00

Probability zero does not mean "never happens", and probability one does not mean "always happens". For example, if you choose a real number uniformly in the interval $[0,100]$, you will choose an integer with probability zero, and a transcendental number with probability one. That doesn't mean there aren't any integers though.

To understand what "probability zero" means, you need to know some measure theory; it means that the measure of that event is zero. Similarly, "probability one" means that the measure of that event is one. This idea is used quite frequently in probability theory, because measure-zero events are often annoying and we want to not think about them.

• To say "you need to know some measure theory" seems a little bit exaggerated. (See my answer posted here.) May 24, 2013 at 13:38
• @MichaelHardy, on the contrary, I think at least a little measure theory is essential to understand these terms; as evidence consider OP's comment to your solution. May 24, 2013 at 14:06
• As you can now see, I have considered it. May 24, 2013 at 14:08
• Your example does not prove anything: probability of choosing an integer is zero, AND you never will choose an integer. So probability zero means never happens. Completely wrong answer -1 May 24, 2013 at 15:30
• @vadim123 give an example of such picked number. May 24, 2013 at 15:35

Throw a dart at a square target, the probability of its landing in any subregion of the square being proportional to the area of the region.

The probability that it fails to land exactly on the diagonal from the upper left corner to the lower right corner is $1$, since the area of that diagonal is zero.

But that doesn't mean every point in the space of points where it could land fails to be on that diagonal. Some points are on the diagonal.

That's the difference.

• If we take a particular event, how does adding the phrase 'with probability 1' change things ? I am trying to relate what you wrote here to my question. Is it the same as asking: What is the difference between P(dart fails to land on diagonal) = 1 and P(dart fails to land on diagonal) = 1 with probability 1. May 24, 2013 at 13:48
• Your question is rather phrased as "$\mathbb P$(dart fails to land on diagonal)$= 1$" versus "dart fails to land on diagonal with probability $1$". It is the same thing. What you wanted to know (based on the original question) was the difference between "the dart will not be on the diagonal, with probability $1$" and "the dart cannot be on the diagonal." May 24, 2013 at 13:53
• There is no difference between "P(dart fails to land on diagonal) = 1" and "dart fails to land on the diagonal with probability 1", but your last phrasing, "P(dart fails to land on diagonal) = 1 with probability 1" is confused. Nobody said a probability is 1 with probability 1. The difference, expressed in your question is between "dart fails to land on the diagonal with probability 1$and "dart fails to land on the diagonal". The former can be proved; the latter cannot. May 24, 2013 at 13:57 • area zero=probability zero. Not 1. Proportional, not backward proportional. -1 May 24, 2013 at 15:33 • @Anixx: Maybe I'm misunderstanding your comment, but -- you seem to have misread the sentence, "The probability that it fails to land exactly on the diagonal [...] is 1, since the area of that diagonal is zero." May 24, 2013 at 15:45 Doing it below with random variables — same idea than in your case (where it s the event `"something happens to that sequence of random variables") The difference between$X=a$and$X=a$a.s. (almost surely, i.e. with probability one) is better understood when you actually see a random variable as what is it: a function from some set$\Omega$equipped with a probability distribution to some set of values$\mathcal{X}$. "$X=a$" means "$\forall\omega\in\Omega,\ X(\omega)=a$", i.e "$\{\omega\in \Omega: X(\omega)=a\}=\Omega$". However, "$X=a$a.s" (or, equivalently,$\mathbb P\{X=a\}$) only means the measure of the set$\{\omega\in \Omega: X(\omega)=a\}$is 1. There might be some$\omega$'s for which$X(\omega)\neq a$; but the measure of the set of such$\omega$'s is$0$(that is, the set of "bad"$\omega$'s is negligible). • For the definition of random variable, you write "X=a" means "∀ω∈Ω, X(ω)=a", i.e "{ω∈Ω:X(ω)=a}=Ω". Isn't it the case that for some values of omega, X(omega) = a and not for all values ? i.e. There are some outcomes whose set is identified by a value 'a'. I am wondering why you wrote 'for all'. May 24, 2013 at 13:42 • The question is essentially this: what is the difference between saying "we can show that$X=a$" (implicitly "all the time") and "we can show that$X=a$w.p.$1$". May 24, 2013 at 13:45$S_n$is itself a random variable. Lets say$\mu$is 0.5.$S_n = 0$is a possible event iff$X_i=0$is possible for every i. That means that$lim_{n\rightarrow ∞}\frac{S_n}{n}$is also a random variable! It is$\mu$with probability one, but there are events for which it is not$\mu\$!