2
$\begingroup$

This paradox must probably already have been answered on math.stackexchange, but I can't find it.

Say we have a standard normally distributed variable $X$ (or any other distribution really).

Then we have an interval $I$ (e.g.$(-1,1)$), for which the probability that $X$ is in $I$ is positive. So when we sample $X$, we could for example get $0.345....$

Yet for any number $r \in \mathbb R$, the probability that $X$ will turn out to be equal to $r$ is $\int_r^rD_X(x)dx=0$ Where $D_X$ is the probability density function of $X$.

Paradox: So for any individual $r\in R$, the probability $P(X=r)=0$. That means that when we sampled $0.345...$ just now, an event occurred that has probability $0$ of occurring.

How is this paradox resolved?

$\endgroup$

closed as unclear what you're asking by Yves Daoust, Henrik, Adam Hughes, E. Joseph, user223391 May 14 '17 at 23:40

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Can you please pinpoint where you see a paradox ? Probability $0$ doesn't mean impossible. $\endgroup$ – Yves Daoust May 12 '17 at 13:41
  • $\begingroup$ This is closely related to the disputed existence of "infinitesimals" used early on to explain the calculus: How can the sum of (infinitely many) infinitely small things give a substantially positive result? One can resolve this either by avoiding the introduction of infinitesimals or by putting the existence and operation of infinitesimals on a rigorous foundation. (The latter can be accomplished in more than one way.) $\endgroup$ – hardmath May 12 '17 at 15:59
  • $\begingroup$ I think this question is not worth closing. To me, it's pretty clear what the question is. It's clear that the derivation of a contradiction comes from the intuition that it's impossible for an event with zero probability to happen. $\endgroup$ – Timothy Jul 23 at 4:53
2
$\begingroup$

$0$ probability doesn't imply an impossible event. If you think of probability of an event as the ratio of number of occurrences of the event to the total number of occurrences, then as the continuous random variable can take uncountably many values, the probability that it is equal to a particular number is negligible or $0$. This is not mathematically rigorous but that is the intuition.

$\endgroup$
  • $\begingroup$ You probably mean impossible, rather than improbable. $\endgroup$ – Yves Daoust May 12 '17 at 13:39
  • $\begingroup$ This seems to me like it's just a reformulation of my paradox? not a solution? $\endgroup$ – user56834 May 12 '17 at 13:40
  • $\begingroup$ sorry corrected it $\endgroup$ – Abishanka Saha May 12 '17 at 13:40
2
$\begingroup$

This paradox is resolved when you realize that finite and infinite probability spaces don't always act the same. In any probability space, for some event $E$, $pr(E)=0$ means that the probability measure of $E$ is zero. In finite spaces we have that this implies that $E$ never happens. This same implication doesn't hold for infinite probability spaces since the union of infinitely many sets of measure zero can have positive measure.

$\endgroup$
  • $\begingroup$ I understand what you are saying, but it seems to me that this doesn't resolve the paradox, but only shifts it a bit. Because the defining feature (intuitively) of a probability of 0, is that it cannot occur. $\endgroup$ – user56834 May 12 '17 at 14:49
  • 1
    $\begingroup$ There is no axiom that says "a probability 0 event cannot occur." $\endgroup$ – Michael May 12 '17 at 14:51
  • $\begingroup$ Maybe I have to change something in the way I think about it, but it seems to me that what a probability of $a$ is, is that it will occur an $a$ fraction of the time. $\endgroup$ – user56834 May 12 '17 at 14:51
  • $\begingroup$ @Programmer2134 that logic only follows in finite spaces. Maybe it will help if we say that "we expect it will occur an $a$ fraction of the time. We expect to never choose some specific number, but just because it is unexpected, that doesn't mean it can't happen. $\endgroup$ – Sean English May 12 '17 at 14:53
0
$\begingroup$

Abishanka's and Sean's intuitions are good and correct. Here are related thoughts:

1) You may as well make your random variable $X$ uniformly distributed over the interval $[0,1]$. Each point is equally likely, there are infinitely many, so the probability of each must be zero. We must integrate the uniform density over an interval in order to accumulate any positive probability.

Same with throwing darts at a dart board: Each point is equally likely but there are infinitely many of them. We can only talk about positive probabilities of the dart falling in a region with positive area.

So if you first pick a particular number $x \in [0,1]$, then $P[X=x]=0$. If you pick a countably infinite set $\mathcal{A} \subsetneq [0,1]$, then $P[X \in \mathcal{A}]=0$. Notice that we are picking the point and/or the set first, and then wondering if we will win the "probability 0 lottery" that $X$ lands on our point or set. On the other hand, we know that some outcome will occur, so $P[X \in [0,1]]=1$.

2) Since $X$ is uniformly distributed over $[0,1]$, its value will be irrational with probability 1. This is described by an infinite decimal precision. Do you really think you could specify such a number in advance? An actual computer simulation would truncate $X$ to finite precision, in which case the approximation $\tilde{X}$ is discrete and all of its (finite number) of outcomes have positive probability.

3) We can only sum over countably infinite sets: The interval $[0,1]$ contains an uncountably infinite number of points. So if we try to obtain contradictions by a summing argument, we will fail, since our summation cannot sum over all possible outcomes.

A related interesting theorem is the following:

Theorem:

Let $B$ be an uncountably infinite set. Let $f:B\rightarrow\mathbb{R}$ be a function with positive values (so that $f(x)>0$ for all $x \in B$). Then there is a countably infinite subset $C \subsetneq B$ such that $\sum_{x \in C} f(x) = \infty$.

Proof:

Let $\mathcal{F}$ be the collection of all finite subsets of $B$. Define $m = \sup_{A \in \mathcal{F}} \sum_{x \in A} f(x)$. Then there is a sequence of finite sets $\{A_n\}_{n=1}^{\infty}$ such that $A_n \in \mathcal{F}$ for all $n$, and $\lim_{n\rightarrow\infty}\sum_{x \in A_n} f(x) = m$. The set $\cup_{i=1}^{\infty} A_i$ is the countable union of a finite sets, so it is either finite or countably infinite. Clearly: $$ \sum_{x \in A_n} f(x) \leq \sum_{ x \in \cup_{i=1}^{\infty} A_i} f(x) \quad, \forall n \in \{1, 2, 3, ...\} $$ and taking limits as $n\rightarrow \infty$ gives $m \leq \sum_{ x \in \cup_{i=1}^{\infty}A_i} f(x)$. If $m=\infty$, then we are done, since we just define the finite or countably infinite set $C = \cup_{i=1}^{\infty} A_i$ and we have $C \subsetneq B$ and $\sum_{x \in C} f(x) = \infty$. (In particular, since the sum is infinite, the "finite or countably infinite" set $C$ cannot be finite, and so it is countably infinite.)

Now suppose $m < \infty$ (we reach a contradiction). Since $\cup_{i=1}^{\infty} A_i$ is a finite or countably infinite subset of the uncountably infinite set $B$, there is a point $y \in B$ such that $y \notin \cup_{i=1}^{\infty} A_i$. Note that $f(y)>0$. Now choose an index $k$ such that $\sum_{x \in A_k} f(x) \geq m-f(y)/2$. Note that $y \notin A_k$. Consider the finite set $A_k \cup \{y\}$. Then $\sum_{x \in A_k \cup \{y\}} f(x) = f(y) + \sum_{x \in A_k} f(x) \geq f(y) + m-f(y)/2 > m$, contradicting the fact that the sum over any finite subset of $B$ must be less than or equal to $m$ (by definition of $m$). $\Box$

In particular, this theorem implies that any probability distribution can have an at-most-countably-infinite number of positive point masses. It also shows why we are only allowed to sum over a countably infinite number of terms: Summing an uncountably infinite number of positive numbers necessarily gives $\infty$.

$\endgroup$
  • $\begingroup$ "Do you really think you could specify such a number in advance?" No I don't, but theoretically, we can. $\endgroup$ – user56834 May 12 '17 at 14:46
  • $\begingroup$ Thus, theoretically, we can have an outcome of a random variable $X$ uniformly distributed over $[0,1]$. So, theoretically, a probability 0 event can occur. $\endgroup$ – Michael May 12 '17 at 14:51

Not the answer you're looking for? Browse other questions tagged or ask your own question.