11
$\begingroup$

Consider X as a continuous random variable which can assume any value in [0, 1]. It is known that P(X=x)=0 where P is the probability density function. I want to understand this intuitively.

The math insight article helps me somewhat:

In other words, the probability that the random number X is any particular number x∈[0,1] (confused?) should be some constant value; let's use c to denote this probability of any single number. But, now we run into trouble due to the fact that there are an infinite number of possibilities. If each possibility has the same probability c and the probabilities must add up to 1 and there are an infinite number of possibilities, what could the individual probability c possibly be? If c were any finite number greater than zero, once we add up an infinite number of the c's, we must get to infinity, which is definitely larger than the required sum of 1. In order to prevent the sum from blowing up to infinity, we must have c be infinitesimally small, i.e., we must insist that c=0.

But, what if I choose, c=1/N (where N is that large number) for the uniform case? The sum will still be 1 as far as I can understand. For the non-uniform case, I can pick some 0's and others non-zeros and still be theoretically able to get a sum of 1 for all the possible values.

Anything that helps me understand this clearly will be of immense help. The answer here helps but I still don't get it.

$\endgroup$
2
  • 1
    $\begingroup$ The probability density function is not zero. The probability is. $\endgroup$
    – aschepler
    May 1, 2015 at 0:02
  • $\begingroup$ Right, that's what I meant by P(X=x)=0 $\endgroup$
    – Amit
    May 1, 2015 at 0:04

3 Answers 3

10
$\begingroup$

A continuous random variable can realise an infinite count of real number values within its support -- as there are an infinitude of points in a line segment.

So we have an infinitude of values whose sum of probabilities must equal one. Thus these probabilities must each be infinitesimal. That is the next best thing to actually being zero. We say they are almost surely equal to zero.

$$\Pr(X=x) = 0 \text{ a.s.}$$

( To have a sensible measure of the magnitude of these infinitesimal quantities, we use the concept of probability density, which yields a probability mass when integrated over an interval. This is, of course, analogous to the concepts of mass and density of materials. )

$$f_X(x) = \frac{\mathrm d}{\mathrm d x}\Pr(X\leq x)$$


For the non-uniform case, I can pick some 0's and others non-zeros and still be theoretically able to get a sum of 1 for all the possible values.

You are describing a random variable whose probability distribution is a mix of discrete (massive) points and continuous intervals. This has step discontinuities in the cumulative distribution function.

$\Pr(X\leq x) = \begin{cases} 0 & x < 0 \\ 0.25 & x=0 \\ 0.25+x/4 & 0< x< 1/2 \\ 0.75 & x=1/2 \\ 0.5+x/4 & 1/2< x< 1 \\ 1 & x\geq 1\end{cases}\\[2ex] \Pr(X=x) = \begin{cases} 0 & x<0 \cup x> 1 \\ 0.25 & x=0 \cup x=1/2 \cup x=1 \\ 0\text{ a.s.} & 0<x<1/2 \cup 1/2<x< 1 \end{cases}$

$\endgroup$
4
  • $\begingroup$ Thank you for the explanation. I like your explanation. However, for the non-uniform case, what if I reframe my question to ask that I can have different infinitesimal values instead of being exactly 0? Will the same reason as uniform probability apply? That, moving to uncountably infinite space, the values are infinitesimal, and for practical purposes, close to 0? $\endgroup$
    – Amit
    May 1, 2015 at 12:35
  • 2
    $\begingroup$ Please see the comments under this question concerning the use of "almost surely" in this answer. $\endgroup$
    – joriki
    Jul 7, 2018 at 13:21
  • $\begingroup$ what a.s. means? $\endgroup$
    – ESCM
    Jun 26, 2019 at 21:26
  • $\begingroup$ "almost surely" @EduardoS. $\endgroup$ Jun 26, 2019 at 22:40
5
$\begingroup$

The situation is easier to formalize when we say that there is no uniform distribution on a countably infinite set $S$. This is because if there were, then $P(X=x)=c>0$ for every $x$, and now

$$P(X \in S)=\sum_{x \in S} c = +\infty.$$

This follows from the property of countable additivity of probability, which is usually treated as an axiom. I think that at least finite additivity of probability should be intuitively obvious, and countable additivity is a straightforward extension.

In the uncountable case, we need one more result, which can be proven from the countable additivity axiom: if $A \subset B$ then $P(A) \leq P(B)$; so now if $S$ is uncountable, and we are to assign a uniform distribution to it, then we can extract a countably infinite subset $C$. Then

$$P(X \in S) \geq P(X \in C) = \sum_{x \in C} c = +\infty$$

as before.

$\endgroup$
3
  • $\begingroup$ Thanks Ian. I will think more about this, I feel I have got a new way of thinking about it. $\endgroup$
    – Amit
    May 1, 2015 at 0:36
  • $\begingroup$ what about the non-uniform case? $\endgroup$
    – Amit
    May 1, 2015 at 0:37
  • $\begingroup$ @Amit In the non-uniform case, if you take for granted that the CDF of a continuous variable is continuous, then the situation is simple: $P(X=x)=\lim_{y \to x^-} F(x)-F(y)=0$. The first equality is general, while the second equality is from continuity. $\endgroup$
    – Ian
    May 1, 2015 at 0:42
1
$\begingroup$

Use the fact that if $A \subset B$, then $p(A) \le p(B)$.

If $X$ is a continuous random variable, this means that the cdf is continuous, that is $f(\alpha) = p(\{\omega | X(\omega) \le \alpha \})$ is continuous.

It follows that $p((a,b]) = f(b)-f(a)$.

Suppose $a < x \le b$. Then we have $p((a,b]) = f(b)-f(a)$.

Since $\{x\} \subset [a,b]$ for all such $a,b$, we have $p(\{x\}) \le f(b)-f(a)$ for all such $a,b$. Hence $p(\{x\})=0$, since $f$ is continuous.

$\endgroup$
6
  • $\begingroup$ Technically, because you are allowing $x=b$, this is essentially begging the question, since $[x,x]=\{ x \}$. But everything works out provided you assume $x<b$ (and separately manage the case $x=1$). $\endgroup$
    – Ian
    May 1, 2015 at 0:06
  • $\begingroup$ Everything works out the way it is above. I don't understand why you want $x<b$? Containment is all that is needed here. $\endgroup$
    – copper.hat
    May 1, 2015 at 0:08
  • $\begingroup$ By saying that $P([a,b])=b-a$ for $a \leq b$, I can just take $a=b=x$ to get $P([x,x])=P(\{ x \})=0$. So you are actually just asserting the claim without justifying it. $\endgroup$
    – Ian
    May 1, 2015 at 0:09
  • $\begingroup$ @cooper.hat sorry, I am looking for an answer with less math, and more intuition. $\endgroup$
    – Amit
    May 1, 2015 at 0:09
  • $\begingroup$ The point was not to compute the probability explicitly, but to show that it must be zero because it is contained in increasingly unlikely events. This was meant to help intuition, not computation. $\endgroup$
    – copper.hat
    May 1, 2015 at 0:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .