# Zero probability and impossibility

I read a comment under this question:

There are plenty of events that can occur that have zero probability.

This reminds me that I have seen similar saying before elsewhere, and have never been able to make sense out of it. So I was wondering

1. if zero probability and impossibility mean the same?
2. if an event with zero probability doesn't mean that the event is impossible to occur, how probability theory represents/describes impossibility?

Thanks and regards!

-

Adding to what others have already mentioned. There is also this notion of plausible event. I am not sure if this is standard. But in the book "Measure Theory and Probability" by Malcolm Ritchie Adams and V. Guillemin, a plausible event is defined as an event which corresponds to a Borel set.

Hence, my understanding of the three words is as follows:

If we take the probability space $(X,\mathscr{F},\mu )$,

An event $A \subseteq X$ is impossible if $A = \emptyset$

An event $A \subseteq X$ is implausible if $A \notin \mathscr{F}$

An event $A \subseteq X$ is improbable if $\mu^*(A) = 0$

-
An implausible event doesn't satisfy our intuitive notion of "improbable", since if an event is implausible then so is its complement. – Yuval Filmus May 24 '11 at 20:48
@Yuval: True. On a slightly similar note, I am not sure intuitively if we would want $\mu(A) = 0$ or $\mu^*(A)$ for an improbable event. – user17762 May 24 '11 at 20:59
Thanks! Just wonder what the star means in $\mu^*(A) = 0$? – Tim May 24 '11 at 20:59
@Tim: By $\mu^*$, I mean the outer measure – user17762 May 24 '11 at 21:01
Why do you use the outer measure instead of the measure to define an improbable event? Is this how the book you cited does? – Tim May 24 '11 at 21:03

Zero probability isn't impossibility. If you were to choose a random number from the real line, 1 has zero probability of being chosen, but still it's possible to choose 1.

-
More information here: en.wikipedia.org/wiki/Almost_surely – F M May 24 '11 at 19:41
One could argue that such a random choice of a real doesn't have zero probability, but only infinitesimal probability. Thus, it looks like you have zero probability, when you don't. – Doug Spoonwood Jun 13 '11 at 13:42
No, Doug. Fernando is correct. Any finite number divided by an infinite number is 0, not an "infinitesimal probability". Dealing with infinities is a difficult concept to grasp, sort of like Einstein's relativity. – lkessler Jul 30 '14 at 16:56
You probably want a bounded interval instead of the real line, assuming you're taking (normalized) Lebesgue measure. – anomaly Jan 17 at 6:17

I think the crux of the matter is what probability actually is. In the Bayesian view, probabilities are measures of (personal) confidence or belief, so it's quite obvious why an event with probability zero is not the same thing as an impossible event. But perhaps this isn't such a satisfactory answer. In the frequentist view, probabilities are the asymptotic frequency of events as the number of independent trials tends to infinity. Here again wee see that something that happens with probability zero is not the same as something impossible; it's just something that happens so infrequently that the numerator in $\dfrac{\text{occurences}}{\text{trials}}$ is dominated by the denominator.

Putting aside such philosophical matters, there's also a technical matter to be discussed here. Under the usual measure-theoretic formulation of probability theory, we have a sample space $\Omega$ and a family $\mathcal{F} \subseteq \mathcal{P}(\Omega)$ of events (measurable subsets of $\Omega$), and the probability of an event $A \in \mathcal{F}$ is its measure $\mathbb{P}(A)$. There is nothing in the axioms of measure theory which say that a non-empty set must have a non-zero measure; and if we interpret $\mathcal{F}$ as the set of all possible events, it's clear that an impossible event is not the same thing as an event of zero probability.

To give a concrete example, consider a random variable $X$ which is uniformly distributed on the interval $[0, 1]$. Although $\mathbb{P}[X \in (a, b)] = b - a$ for all $(a, b) \subset [0, 1]$, the axioms of probability force us to conclude that $\mathbb{P}[X = x] = 0$ for any individual $x \in [0, 1]$: for if $\mathbb{P}[X = x] = \epsilon > 0$, because $X$ is uniformly distributed, by additivity of the probabilities of disjoint events, we'd be forced to conclude that $[0, 1]$ contains at most $\frac{1}{\epsilon}$ (a finite number!) points, which is absurd.

-
Thanks! "if we interpret F as the set of all possible events, it's clear that an impossible event is not the same thing as an event of zero probability", (1) do you mean that the empty set is a possible event, and not an impossible one? (2) Are impossible events F-measurable? – Tim May 24 '11 at 20:12
@Tim: I'd say $\emptyset$ is a possible event: it's possible nothing happens. The idea that $\emptyset$ is impossible comes from the fact that an impossible event is outside the scope of the model, whether it be because the "corresponding" sample point is not in $\Omega$ (whatever that means) or because the combination of sample points in question is not measurable. In either case it's meaningless to ask about the probability of such an event because it's undefined. – Zhen Lin May 24 '11 at 20:54
I like the first paragraph, it defines what exactly P(A) = 0 is. – Studnet T Aug 14 '14 at 2:12

Mathematicians generally formalize probability using the notion of a probability space and measure theory. In this formalism it is possible for an event to have probability $0$ without being the empty event. Perhaps the simplest "realistic" (and I use the word loosely) example of such an event is the event of flipping only heads infinitely many times. This event has probability $0$, but it is not empty, which is what one might call a formal definition of "impossible."

The underlying probability space is the set of possible ways to flip a coin infinitely many times. An example of an impossible event here is that you flip, say, cat. The coin has only a heads side and a tails side; it doesn't have a cat side, so flipping cat is impossible.

(Whether this formalism says anything reasonable about the real world is debatable. In practice, events of sufficiently small probability are already impossible. The above is just a statement about a certain mathematical formalism that has proven to be useful in certain contexts. In mathematics, we want to prove statements about some class of objects. Sometimes we can prove that the statement holds with probability $1$, but this does not imply that it holds for all objects, and since we actually care about all objects this distinction really does need to be made in mathematics.)

-
"In practice, events of sufficiently small probability are already impossible." Not true! You give me a non-zero probability, however small, and I can generate an event on my laptop which has an even smaller probability of occurring. I just have to generate enough random bits. For this, I only need time proportional to the logarithm of the reciprocal of your probability. (Of course you could make this reciprocal logarithm really big, but I would still win for all reasonable definitions of "really big".) – TonyK May 24 '11 at 19:50
That feels like a cop-out. Unless you can predict what bits you're going to generate in advance... in practice, you should ignore any event which is less likely than, say, winning the lottery ten times (given that you don't currently play the lottery and you believe yourself to be a rational actor). – Qiaochu Yuan May 24 '11 at 19:52
Thanks! Concerning my second question, do you mean that probability theory represents impossibility with the empty set, not a non-empty set with probability zero? Is this how probability theory distinguishes zero probability and impossibility? – Tim May 24 '11 at 19:54
@Qiaochu: Perhaps we're talking at cross-purposes here. All I'm saying is this: If you ask me to generate an event with probability (say) $2^{-n}$, then all I have to do is generate $n$ random bits. And then an event of probability $2^{-n}$ has occurred. – TonyK May 24 '11 at 19:57
@Tim: yes. @TonyK: but any particular event that you experience is an event of negligible probability in the appropriate probability space. That still feels like a cop-out to me. I'm talking about how low you have to predict the probability of an event to be before you should ignore it for the purposes of making decisions in a practical setting. – Qiaochu Yuan May 24 '11 at 20:00

Let $A$ be an event, $\Pr$ be the probability measure.

$A$ has zero probability if $\Pr(A) = 0$.

$A$ is impossible if $A=\emptyset$.

Impossibility implies zero probability, but the reverse is false. Consider the real line $\mathbb{R}$; if you randomly select a number $x$, the probability that $x=0$ is $0$, but this is not impossible. In fact, the probability that $x$ belongs to some countable set, e.g $\mathbb{Q}$, is also $0$.

From a purely mathematical point of view, impossibility is simply a stronger statement, so impossibility cannot be described by probability measure. However, another way of thinking might shed some light. That is, if the probability that something exists has probability greater than $0$, then it exists. This notion has been used for some mathematical arguments.

-
Thanks! Having said that A is impossible if A is the empty set, how come that impossibility cannot be described by probability measure? Isn't A being the empty set how probability measure describes impossibility? – Tim May 24 '11 at 20:02
What I mean is that $\Pr(A) = 0$ does not imply $A=\emptyset$, i.e. knowing probability measure $=0$ does not help you figure out if the set is empty or not. – Hawii May 24 '11 at 20:27
I like the definition of "impossibility cannot be described by probability measure." – Studnet T Aug 14 '14 at 2:15
So I guess this means the notion of impossibility is external to the 'probabilistic way of thinking' (as it's described here) since whether an event is empty depends on the particular representation (i.e., probability space) used? – Vandermonde Jan 8 at 4:05

Probability theory is an abstract subject, which is not limited to the real world. In cases where it is limited to the real world, an event of zero probability will not occur. But the abstract underpinning of the real-world cases allows for the occurrence of zero-probability events; when you translate these abstract events into events that are physically detectable, their probabilities become non-zero.

-
Thanks! Do you mean that probability theory may not perfectly model the real world, which is the reason that an event assumed to have zero probability in probability theoretical model can have non-zero probability in the real world and therefore can occur in the real world? – Tim May 24 '11 at 19:58
@Tim: the culprit isn't probability theory in general (here). It's the specific information that you have. If your information is incomplete, your assessment of the probability of various events is necessarily going to be incomplete. Your assessment of the probability of an event is (if you subscribe to the appropriate philosophy of probability) a statement about your mind, not about the world. – Qiaochu Yuan May 24 '11 at 20:17
Another way to think about it is that, in the real world, you only have finite probability spaces. The is no "real" way to pick a random number uniformly in $[0,1]$, only to pick successively closer approximations of a real number in $[0,1]$ so that the limit is uniform. Then probability on infinite spaces is about limits of probabilities on finite spaces. This turns out to be useful in the real world because we are dealing with finite spaces so large that the probabilities behave very close to their limits. – Thomas Andrews May 24 '11 at 20:39
@Tim: I mean that probability theory can model the real world (as far as we know), but that it can do more if we want it to. – TonyK May 27 '11 at 11:07