12
$\begingroup$

From a textbook on probability on the Law of Large Numbers:

Theorem 3-19 (Law of Large Numbers): Let $X_1,X_2, \ldots , X_n$ be mutually independent random variables (discrete or continuous), each having finite mean and variance. Then if $S_n = X_1 + X_2 +\dots+ X_n$,

$$ \lim_{n \to\infty} P\left(\left|\frac{S_n}{n} - \mu\right| \geq \varepsilon\right) = 0 $$

Since $S_n$ is the arithmetic mean of $X_1,X_2, \ldots , X_n$, this theorem states that the probability of the arithmetic mean $\frac{S_n}{n}$ differing from its expected value $\mu$ by more than $\varepsilon$ approaches zero as $n \to \infty$. A stronger result, which we might expect to be true, is that $ \lim_{n \to\infty} \frac{S_n}{n} = \mu $ but this is actually false. However, we can prove that $ \lim_{n \to\infty} \frac{S_n}{n} = \mu $ with probability one.

The only difference between the last sentence and the one before that is the phrase 'with probability one'. What does probability one mean here ? The usual definition is that the event occurs with 100% certainty. If that is the case, why is the original assertion $ \lim_{n \to\infty} \frac{S_n}{n} = \mu $ false ?

Improve this question
$\endgroup$
  • 7
    $\begingroup$ If you want to understand how to use $\TeX$ well, take a look at my edits to your question. I changed $lim_{n->\infty}$ to $\lim_{n\to\infty}$ and $\displaystyle lim_{n->\infty}$ to $\displaystyle\lim_{n\to\infty}$ in several instances, and $(|\frac{S_n}{n}-\mu|)$ to $\displaystyle\left(\left|\frac{S_n}{n}-\mu\right|\right)$, and $X_1+X_2 . . . X_n$ to $X_1+X_2+\dots+X_n$, and changed various other things to standard $\TeX$ usage. $\endgroup$ – Michael Hardy May 24 '13 at 13:44
  • 2
    $\begingroup$ Essentially a duplicate of Zero probability and impossibility, only asking about $1$ instead of $0$.. $\endgroup$ – BlueRaja - Danny Pflughoeft May 24 '13 at 16:00
  • 1
    $\begingroup$ The first thing you need to understand is that a Schaum's Outline is not a "textbook." $\endgroup$ – user 726941 May 24 '13 at 17:46
22
$\begingroup$

Probability zero does not mean "never happens", and probability one does not mean "always happens". For example, if you choose a real number uniformly in the interval $[0,100]$, you will choose an integer with probability zero, and a transcendental number with probability one. That doesn't mean there aren't any integers though.

To understand what "probability zero" means, you need to know some measure theory; it means that the measure of that event is zero. Similarly, "probability one" means that the measure of that event is one. This idea is used quite frequently in probability theory, because measure-zero events are often annoying and we want to not think about them.

Improve this answer
$\endgroup$
  • 4
    $\begingroup$ To say "you need to know some measure theory" seems a little bit exaggerated. (See my answer posted here.) $\endgroup$ – Michael Hardy May 24 '13 at 13:38
  • 1
    $\begingroup$ @MichaelHardy, on the contrary, I think at least a little measure theory is essential to understand these terms; as evidence consider OP's comment to your solution. $\endgroup$ – vadim123 May 24 '13 at 14:06
  • $\begingroup$ As you can now see, I have considered it. $\endgroup$ – Michael Hardy May 24 '13 at 14:08
  • $\begingroup$ Your example does not prove anything: probability of choosing an integer is zero, AND you never will choose an integer. So probability zero means never happens. Completely wrong answer -1 $\endgroup$ – Anixx May 24 '13 at 15:30
  • $\begingroup$ @vadim123 give an example of such picked number. $\endgroup$ – Anixx May 24 '13 at 15:35
8
$\begingroup$

Throw a dart at a square target, the probability of its landing in any subregion of the square being proportional to the area of the region.

The probability that it fails to land exactly on the diagonal from the upper left corner to the lower right corner is $1$, since the area of that diagonal is zero.

But that doesn't mean every point in the space of points where it could land fails to be on that diagonal. Some points are on the diagonal.

That's the difference.

Improve this answer
$\endgroup$
  • $\begingroup$ If we take a particular event, how does adding the phrase 'with probability 1' change things ? I am trying to relate what you wrote here to my question. Is it the same as asking: What is the difference between P(dart fails to land on diagonal) = 1 and P(dart fails to land on diagonal) = 1 with probability 1. $\endgroup$ – curryage May 24 '13 at 13:48
  • 1
    $\begingroup$ Your question is rather phrased as "$\mathbb P$(dart fails to land on diagonal)$ = 1$" versus "dart fails to land on diagonal with probability $1$". It is the same thing. What you wanted to know (based on the original question) was the difference between "the dart will not be on the diagonal, with probability $1$" and "the dart cannot be on the diagonal." $\endgroup$ – Clement C. May 24 '13 at 13:53
  • 2
    $\begingroup$ There is no difference between "P(dart fails to land on diagonal) = 1" and "dart fails to land on the diagonal with probability 1", but your last phrasing, "P(dart fails to land on diagonal) = 1 with probability 1" is confused. Nobody said a probability is 1 with probability 1. The difference, expressed in your question is between "dart fails to land on the diagonal with probability 1$ and "dart fails to land on the diagonal". The former can be proved; the latter cannot. $\endgroup$ – Michael Hardy May 24 '13 at 13:57
  • $\begingroup$ area zero=probability zero. Not 1. Proportional, not backward proportional. -1 $\endgroup$ – Anixx May 24 '13 at 15:33
  • $\begingroup$ @Anixx: Maybe I'm misunderstanding your comment, but -- you seem to have misread the sentence, "The probability that it fails to land exactly on the diagonal [...] is 1, since the area of that diagonal is zero." $\endgroup$ – ruakh May 24 '13 at 15:45
4
$\begingroup$

Doing it below with random variables — same idea than in your case (where it s the event `"something happens to that sequence of random variables")

The difference between $X=a$ and $X=a$ a.s. (almost surely, i.e. with probability one) is better understood when you actually see a random variable as what is it: a function from some set $\Omega$ equipped with a probability distribution to some set of values $\mathcal{X}$.

"$X=a$" means "$\forall\omega\in\Omega,\ X(\omega)=a$", i.e "$\{\omega\in \Omega: X(\omega)=a\}=\Omega$".

However, "$X=a$ a.s" (or, equivalently, $\mathbb P\{X=a\}$) only means the measure of the set $\{\omega\in \Omega: X(\omega)=a\}$ is 1. There might be some $\omega$'s for which $X(\omega)\neq a$; but the measure of the set of such $\omega$'s is $0$ (that is, the set of "bad" $\omega$'s is negligible).

Improve this answer
$\endgroup$
  • $\begingroup$ For the definition of random variable, you write "X=a" means "∀ω∈Ω, X(ω)=a", i.e "{ω∈Ω:X(ω)=a}=Ω". Isn't it the case that for some values of omega, X(omega) = a and not for all values ? i.e. There are some outcomes whose set is identified by a value 'a'. I am wondering why you wrote 'for all'. $\endgroup$ – curryage May 24 '13 at 13:42
  • $\begingroup$ The question is essentially this: what is the difference between saying "we can show that $X=a$" (implicitly "all the time") and "we can show that $X=a$ w.p. $1$". $\endgroup$ – Clement C. May 24 '13 at 13:45
4
$\begingroup$

$S_n$ is itself a random variable. Lets say $\mu$ is 0.5. $S_n = 0$ is a possible event iff $X_i=0$ is possible for every i. That means that $lim_{n\rightarrow ∞}\frac{S_n}{n}$ is also a random variable! It is $\mu$ with probability one, but there are events for which it is not $\mu$!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.