7
$\begingroup$

Can an observed event in fact be of zero probability?

Of course, I know that there exist non-empty events of zero probability. Mu question is the reverse: given that we have observed an event (and we have no other information about it, just the fact that it has been observed), is it possible that the event has in fact zero probability? Or does an observation necessarily mean that the probability is strictly positive?


Example context for the question:

Suppose $x_i$ (countably many) are i.i.d on $[0,1]$, but we do not know the distribution they come from. It may be uniform on $[0,1]$, may be discrete, may be any legitimate distribution (discrete or continuous). Just imagine we have some sort of a machine that shows us one by one a list of randomly drawn numbers between $0$ and $1$. We are comparing the observed numbers one by one to some special number chosen beforehand, for example $\frac12$. Now, given that at some iteration we have observed that special number at least once, does that mean that $\frac12$ has some positive probability under that (unknown) distribution? And if the probability can be zero, can we nevertheless say that we will necessarily observe $\frac12$ again later, if we continue the experiment ad infimum?

Also, disregard the "real world limitations" such as an inability to produce truly uniformly distibuted numbers, or rounding errors or any such thing.

$\endgroup$
7
  • 7
    $\begingroup$ Under continuous probability models, it depends on how you interpret your observation. The probability that a randomly chosen person weighs $100\pi$ pounds is $0$. (We chose $\pi$ for its association with roundness.) But a real weight measurement, because of imprecisions, really should be reported as an interval, and the probability of lying in that interval is non-zero. $\endgroup$ Commented Sep 5, 2014 at 8:44
  • $\begingroup$ I would add another interpretation, that is that a single exact event that is observed is probably a zero probability event, in the exact fashion it happened, due to the number of particules involved in most events... Depends on your definition of "event" and "observed" ;-), since you did not mention any "result". $\endgroup$
    – Martigan
    Commented Sep 5, 2014 at 13:23
  • $\begingroup$ I have added some context to the question. Please see if that answers yours. $\endgroup$
    – Aahz
    Commented Sep 5, 2014 at 13:35
  • 1
    $\begingroup$ If you don't know the probability distribution for your (i.i.d.) random variables, then you do not have a probability theory question. Period. And as Did remarked, in case the distribution should be uniform, then every possible observed value (assumed infinitely precise) has zero probability. And will continue to do so after it has been observed (this is the "independent" in i.i.d.). $\endgroup$ Commented Sep 5, 2014 at 15:25
  • $\begingroup$ Almost never $\endgroup$
    – James
    Commented Sep 5, 2014 at 16:00

3 Answers 3

14
$\begingroup$

is it possible that the event has in fact zero probability?

Yes. For a situation where this always happens, assume that one observes a random number $x$ drawn from the uniform distribution on $(0,1)$. Then the probability to observe $x$ is zero.

(Proof: For every interval $I\subseteq(0,1)$ the probability to observe a number in $I$ is the length of $I$. For every positive $\varepsilon$, there are intervals $I\subseteq(0,1)$ which contain $x$ and have length less than $\varepsilon$ hence the probability to observe exactly $x$ is less than $\varepsilon$, for every positive $\varepsilon$, QED.)

Edit: In a discrete space $\{x_i\mid i\in\mathbb N\}$, the probability to observe $x_i$ is, by hypothesis, some positive $p_i$ hence the above applies only to continuous distributions (or at least, partially continuous).

$\endgroup$
14
  • 1
    $\begingroup$ I have thought of this example before writing. Here's what confuses me: the event of probability zero is not to observe some number (we see some number with probability one). The event with zero probability is predicting the number you draw. I.e. saying "the next number I draw will be $0.5$", and then drawing $0.5$. $\endgroup$
    – Aahz
    Commented Sep 5, 2014 at 8:50
  • $\begingroup$ can it be extended to discrete spaces? $\endgroup$
    – Alex
    Commented Sep 5, 2014 at 8:51
  • 1
    $\begingroup$ Yes: as you say in your question, "given that we have observed an event (and we have no other information about it, just the fact that it has been observed)", in the situation I describe, the event to observe exactly the value observed has zero probability. (Of course, this implicitly assumes that, in real life, one can choose uniformly a random number in (0,1), something which might be debatable). $\endgroup$
    – Did
    Commented Sep 5, 2014 at 8:54
  • $\begingroup$ @Alex "can it be extended to discrete spaces? " No, see Edit. (The "Yes" in my previous comment refers to the OP's "Here's what confuses me" comment.) $\endgroup$
    – Did
    Commented Sep 5, 2014 at 9:03
  • 1
    $\begingroup$ @Aahz No, the fact that you observed it once doesn't mean you'll observe it again later. $\endgroup$
    – yo'
    Commented Sep 5, 2014 at 10:46
4
$\begingroup$

Your question is not a mathematical question, since it depends on the interpretation of probability. It also depends on precisely what you mean by "given that we have observed...".

For example, subjective Bayesians think of probability theory as theory of how (idealized) rational agents should update their beliefs in light of new observations. It is part of this idealized framework that when an event is observed, the probability measure gets updated in such a way that the observed event gets probability 1. IOW, the a posteriori probability of an observed event is not only non-zero, but equal to 1. On the other hand, the a priori probability (the probability before the update) could be anything. Normally, Bayesians adopt additional rationality principles such as always assigning non-zero a priori probabilities to non-empty events in finite sample spaces. But modelling idealizations or misjudgements about the appropriate sample space might still amount to assigning a zero probability to events that are possible.

In an experiment where a sugar cube with faces labeled 1,2,3,4,5,6 is thrown exactly once onto a wet surface, Bob might choose the sample space {1,2,3,4,5,6} and Alice might choose the sample space {1,2,3,4,5,6,edge}. If the sugar cube ends up balancing on an edge, Alice could update her probabilities using Bayes theorem (if she assigned edge some small but non-zero probability). Bob, on the other hand, would've observed an event that he had effectively assigned a vanishing a priori probability. So he would need to update using some other rule than Bayes rule, since he also needs to expand his sample space.

There are many other interpretations of probability than subjective Bayesianism, and the detailed answer to your question may vary depending on which interpretation you prefer. This is why it is not a mathematical question.

EDIT: Regarding the added clarification to the original question, it is certainly possible (and typical!) to have a zero probability of $x_i = 1/2$ in the continuous case. In the discrete case, where the probability mass is concentrated on some finite or countably infinite discrete subset of [0,1], I am not sure. If, say, one has observed that $x_1 = \frac{1}{2}$, then intuitively it does seem plainly irrational to base predictions about $x_k$, $k > 1$, on a probability measure that assigns $P(x_k=\frac{1}{2}) = 0$. But I don't have any formal argument to support this...

$\endgroup$
12
  • $\begingroup$ I have made an edit to my question, with some context. Please see, if it makes it more precise. $\endgroup$
    – Aahz
    Commented Sep 5, 2014 at 13:37
  • $\begingroup$ This seems to disqualify on principle every probability question as non mathematical. A little too inclusive, maybe? $\endgroup$
    – Did
    Commented Sep 5, 2014 at 13:43
  • $\begingroup$ "In the discrete case", we know pretty well what happens. $\endgroup$
    – Did
    Commented Sep 5, 2014 at 21:49
  • 1
    $\begingroup$ OK, I see what you meant has probability 1 now. What is your formal argument for "If p is discrete and one observes x1=x then p(x)>0"? Test case for formal arguments: Let Y be a continuous random variable uniformly distributed on [0,1]. For any arbitrary value, say 1/3, we may define the discrete variable Z = 1 if Y = 1/3 and Z = 0 otherwise. Now, observing Y=1/3 does not mean that P(Y=1/3) > 0. Equivalently, observing Z=1 does not mean that P(Z=1) > 0. PS. Maybe I should stop now, apologies to everyone for the extended exchange. $\endgroup$ Commented Sep 5, 2014 at 23:11
  • 1
    $\begingroup$ p(Z) is discrete. Even forgetting about this kind of examples, it's genuinely unclear to me how a positive formal argument would go. $\endgroup$ Commented Sep 6, 2014 at 0:05
1
$\begingroup$

If I understand your question correctly, this is impossible due to Bayes rule: $$ P(V|S) = \frac{P(S|V)P(V)}{P(S)} $$ where $V$ is the event. Clearly LHS is greater than zero (let's say $\epsilon$) because you have observed it. If its probability is 0 though, then $P(V)=0$ and RHS is 0, and $\epsilon \neq 0$.

$\endgroup$
2
  • 1
    $\begingroup$ If I'm not mistaken the Bayes rule in general is problematic when you deal with continuous distributions, so relying on it may not be entirely correct. $\endgroup$
    – Aahz
    Commented Sep 5, 2014 at 8:55
  • $\begingroup$ That's why I asked Did about it - this seems to apply to discrete spaces, but not continuous ones. $\endgroup$
    – Alex
    Commented Sep 5, 2014 at 8:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .