# Problems with definition of almost surely

Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space. In probability theory one says that and event $F\in\mathcal F$ happens $\mathbb P$-almost surely, if $\mathbb P(E)=1.$ Intuitively, as a beginner one thinks that this means that there exists an event $N\in\mathcal F$ with $\mathbb P(N)=0$ and such that the event $E\setminus N$ happens surely? Is this intuition generally true? Counterexample?

• What does it mean to happen surely? That $E\setminus N = \Omega$? certainly this cannot happen unless $E$ already happened surely because $E\setminus N \subseteq E$. Nov 12, 2013 at 22:57
• You probably wanted to say that $E \cup N$ happens surely. If so, then yes, that is generally true. Just take $N = \Omega \setminus E$. Nov 12, 2013 at 23:00
• what I mean by surely is, that it happens for sure, i.e. we know that it already happend (not sure about this) Nov 12, 2013 at 23:00
• @Feanor: ok, I see your point. Nov 12, 2013 at 23:04
• I'm not sure about it. It depends on what intuition you have, obviously. I think of $\Omega$ as the set of all things that could, in principle, happen. Then, if I was to exclude at least one possibility (i.e. element of $\Omega$), then the resulting "event" would not be sure - because there would be at least one possibility for it not to happen. Nov 12, 2013 at 23:20

In many probability spaces there are events $E \subsetneq \Omega$ and $\varnothing \neq N \subset \Omega$ such that $P(E)=1$ and $P(N)=0$.

Take for example the following scenario:

Suppose you flip a fair coin infinitely many times. The sample space $\Omega$ will be all sequences which look like $(x_1, x_2, x_3, …)$ where $x_i = 0$ if you flip a tail and $x_i= 1$ if you flip a heads. For instance, if the first two flips were tails and the third a heads, you have a sequence starting with $(0,0,1,x_4, …)$. Now, let's ask: What is the probability that the coin flipped all heads? The event we want to look at is $N = \{(1,1,1,1,1,1,…)\}$ and $$P(N) = \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{2} \cdots = \lim_{n \to \infty}\left(\frac{1}{2}\right)^n = 0.$$ This intuitively makes sense because how likely are you to actually flip all heads with a fair coin and infinitely many flips? But, you can use this same strategy and see that the probability of any one fixed sequence of coin flips must be $0$. But again, this intuitively makes sense because how likely are you to flip exactly the same faces of the coin in exactly the same order as the fixed sequence for infinitely many flips?

Now, if I ask what is the probability that you flip a heads on your first flip, lets call this event $H$, the answer is sensibly $P(H)=\frac{1}{2}$. So how can we merge a positive probability of events like $H$ when we have that each single outcome has probability $0$? This is the challenge that measure theory has come to answer. If $P$ is that measure (the measure of probability) and now you ask what is the probability of the event $E$ that you don't flip all heads? Well, the probability of flipping all heads is $0$, so $P(E) =1$. But $E \neq \Omega$ since $E = \Omega \backslash \{(1, 1, 1, 1, …)\}$. Therefore, can we say that if you flip a coin infinitely many times, then you will surely have an outcome that lands in $E$? No! It could so happen that you do flip all heads. Although, since the probability of that happening is $0$ (and just based on our intuition of how likely that is), you will almost surely flip a sequence that lands in $E$.

• @AndyTeich ... copy/pasting ... I think you are confusing "sure" in terms of Ω (i.e. set) and "almost sure" in terms of P (i.e. measure). These are slightly different things ... Nov 12, 2013 at 23:49
• @rtybase: I think I am not confusing them, but my question is lying at the heart of this "slight difference" you are mentioning! Nov 12, 2013 at 23:51
• @AndyTeich You tell me! If you remove even more zero-probability collections of sequences, and then flipped a coin, how could you be more sure than before when just removing the one sequence with all heads? The only event which you can be sure contains all outcomes of the coin flip is the collection of all possible outcomes, which is $\Omega$. Since $E\backslash N \subset E \subsetneq \Omega$, there is nothing I can remove from $E$ that will "make up for" the outcomes that are already absent in $E$. Although, I can still be almost sure that $E\backslash N$ will happen.
– Tom
Nov 12, 2013 at 23:54
• Well ... "sure" is related to "set" which is $\Omega$, "almost sure" is related to "measure" as a tool to measure "sets" (in sigma algebras) which is $P$. Since "set" and "measure" are different entities/tools, then it's quite cumbersome trying to combine them, thus the confusion ... I suspect. If it helps, think of this this way, because $P(\Omega)=1$ then "sure" implies "P-almost sure", vice-versa doesn't always work (depends on sigma algebra's construction). Nov 12, 2013 at 23:55
• @Tom: I think I got it. Your example shows that the only sure thing is the set of all sequences of head and tails, i.e. intuitively it means one of this sequences must appear if we do the experiment. But for any proper subset, it is not true that one of the sequences in the subset must appear (because a sequence from the complement could appear), which is why we cannot consider any subset to be a sure event! Nov 12, 2013 at 23:58