3
$\begingroup$

My Question

For continuous random variables / continuous distributions, it is defined that the probability of any point has probability $0$. The most common proof for this is as follows:

$$\Pr(X=a)=\Pr(a\leq X\leq a) = \int_a^af(x) \, dx=0$$

I am looking for a proof other than this. Below I included the definition of a continuous distribution and the three axioms of probability. Following that I give my attempt at the proof. Please comment with suggestions / advice.

Axioms of Probability

Axiom 1: For every event $A$, $\Pr(A)\geq 0$

Axiom 2: $\Pr(S)=1$

Axiom 3: $\Pr\left(\bigcup\limits_{i=1}^\infty A_i\right) =\sum\limits_{i=1}^\infty \Pr(A_i)$

Definiton of a Continuous Distribution / Random Variable (Degroot & Schervish)

We say that a random variable $X$ has a continuous distribution if there exists a nonnative function $f$, defined on the real line, such that for every interval of real numbers (bounded or unbounded), the probability that $X$ takes a value in the interval is the integral of $f$ over the interval.

My attempt at a proof

It is given that real numbers in there interval $[0,1)$ are uncountable and so every larger set is a fortiori uncountable.

By axiom 2 it follows that $\Pr(S)=1=\Pr(-\infty \leq x\leq\infty) = \int\limits_{-\infty}^\infty f(x) \, dx$

With discrete distributions axiom 3 holds for a countably many infinite number of events. However as a continuous distribution is defined on the real line, and the interval of real numbers in $(-∞,∞)$ is by definition uncountable, the infinite sum of uncountably many different points with mass other than zero would exceed one. Thus they must be zero.

I feel like I am missing something...

$\endgroup$
  • 1
    $\begingroup$ "infinite sum of uncountably different points" What exactly do you mean by that? None of the axioms mentions such a thing. Secondly, what you are describing as "continuous" is actually "absolutely continuous with respect to the Lebesguemeasure". A distribution is continuous if its corresponding CDF is continuous. Not every continuous distribution is absolutely continuous. $\endgroup$ – drhab Nov 27 '15 at 17:50
  • 2
    $\begingroup$ The most common proof is not what you are saying. It is: $P(X=a)\leq (P(a-\epsilon<X\leq a)=F(a)-F(a-\epsilon)$ for every $\epsilon>0$ combined with the fact that $\lim_{\epsilon\to0+} F(a-\epsilon)=F(a)$ since CDF $F$ is continuous. This implies that $P(X=a)=0$. $\endgroup$ – drhab Nov 27 '15 at 17:56
  • $\begingroup$ @drhab I figured out your suggested proof independently yesterday. However, this is not much different from the integral over a point proof as we are essentially taking the integral from 0 to a and subtracting 0 to some $a-\epsilon$ as $\epsilon$ approaches zero we approach some point a. How would you prove this CDF proof in terms of the three axioms of probability? $\endgroup$ – SumNeuron Nov 28 '15 at 19:05
  • 1
    $\begingroup$ Let $a_{n}$ be strictly increasing and converging to $a$. Then $(-\infty,a)=(-\infty,a_{1}]\cup(a_{1},a_{2}]\cup(a_{2},a_{3}]\cup\cdots$ so that $P(X<a)=F(a_{1})+\sum_{n=1}^{\infty}(F(a_{n+1})-F\left(a_{1}\right))=\lim_{k\to \infty}F(a_{k})=F(a)$. The last equality is based on the continuity of $F$ at $a$. Then $P(X=a)=P(X\leq a)-P(X<a)=F(a)-F(a)=0$. $\endgroup$ – drhab Nov 28 '15 at 21:20
  • 1
    $\begingroup$ There is a typo in my former commen: $F(a_{n+1})-F(a_1)$ must be changed into: $F(a_{n+1})-F(a_n)$. $\endgroup$ – drhab Nov 29 '15 at 7:58
1
$\begingroup$

Recall the definition of the distribution function $F$ of a random variable $X$ - for each $x\in\mathbb R$, $$F(x) = \mathbb P(X\leqslant x).$$ Since $F$ is càdlàg, i.e. right-continuous with left limits everywhere, the proper definition of $\mathbb P(X=x)$ would be $$\mathbb P(X=x) = F(x) - \lim_{t\uparrow x}F(t), $$ commonly written as $F(x)-F(x-)$. Now, a continuous random variable has a continuous distribution function, so $F(x-)=F(x)$ for all $x$. Therefore $\mathbb P(X=x)=0$ for all $x$.

$\endgroup$
  • $\begingroup$ thanks for your answer. I came across a similar solution only without the terminology of càdlàg. I appreciate it! $\endgroup$ – SumNeuron Nov 28 '15 at 18:56
0
$\begingroup$

The definition of "continuous distribution" given in de Groot & Schervish is seen in many books. Another reasonable definition conflicting with it is sometimes seen, namely, that the c.d.f. is a continuous function.

The Cantor distribution, whose support is within the interval $[0,1]$, can be defined as follows: the base-$3$ digits of a number $X$ between $0$ and $1$ are i.i.d., each being equal to $0$ or $2$ with probability $1/2$ each. The c.d.f. of this distribution is the Cantor function, which is continuous; this distribution concentrates probability $0$ at each point. But this distribution is not continuous according to the definition in de Groot & Schervish. To see that, $\Pr(X\le1/3 \text{ or } X\ge2/3) = 1$, i.e. the support is entirely within the upper and lower thirds of the interval, a set having measure $2/3$. Then notice that it's entirely within the upper and lower thirds of those two intervals, so it's within a set of measure $(2/3)^2$. Then notice that it's within the upper and lower thirds of those four intervals, so it's within a set whose measure is $(2/3)^3$. And so on: the total measure of the support is $\le(2/3)^n$ for every positive integer $n$. Thus the measure of the support is $0$, and thus the integral of any function over that set is $0$. Thus there can be no density function. But there are also no point masses.

What you are trying to prove is that there are no point masses if the distribution is continuous. If you insist on following the definition in de Groot & Schervish, which defines "continuous" as meaning there's a density function, then I can't see that you can do anything other than observing that the integral of the density over a set containing only one point must be zero.

Can we prove that there are no point masses by the other definition, which allows a somewhat broader class of distributions to be called "continuous" (including the Cantor distribution)? This other definition is simply that the c.d.f. $F$ is a continuous function. Let's prove that $\Pr(X=x)=0$ if $F_X$ is continuous: Continuity at $x$ means $$ \forall\varepsilon>0\ \exists\delta>0\ \forall w\in(x-\delta,x+\delta)\ |F_X(w)-F_X(x)|< \varepsilon. $$ From the last inequality it follows that $F_X\left(x+\frac\delta2\right) - F_X(x) < \varepsilon$ and $F_X(x) - F_X\left(x-\frac\delta2\right) <\varepsilon$; hence $F_X\left(x-\frac\delta2\right) - F_X\left(x+\frac\delta2\right) <2\varepsilon$.

Consequently $\Pr(X=x)<2\varepsilon$.

We now have $\forall\varepsilon>0\ \Pr(X=x)<2\varepsilon$.

The only non-negative number that is less than every positive number (i.e. less than every possible value of $2\varepsilon$) is $0$. Therefore $$ \Pr(X=x)=0. $$

$\endgroup$
  • $\begingroup$ I have a few questions. I don't quite understand what the Cantor distribution is from your description. Base-3 digits of a number between 0 and 1? Each being 0 or 2? Why can they only be 0 or 2? $\endgroup$ – SumNeuron Nov 28 '15 at 18:57
  • $\begingroup$ also is this a typo? $w\in(x-\epsilon,x+\epsilon)$ should it not be $w\in(x-\delta,x+\delta)$ Else why is it $F_X(x+\frac{\delta}{2}$ $\endgroup$ – SumNeuron Nov 28 '15 at 19:00
  • $\begingroup$ @SumNeuron : Sorry --- that was a typo; I've fixed it. ${}\qquad{}$ $\endgroup$ – Michael Hardy Nov 28 '15 at 21:02
  • $\begingroup$ For $n=1,2,3,\ldots$, the $n$th independent coin toss decides whether the $n$th base-$3$ digit of $X$ is $0$ or $2$. That means you have probability $1/2$ that $X\in[0,1/3]$ and probablity $1/2$ that $X\in[2/3,1]$, and probability $0$ that $X\in(1/3,2/3)$. Given that $X\in[0,1/3]$, the conditional probability that $X\in[0,1/9]$ is $1/2$ and that $X\in[2/9,1/3]$ is $1/2$, etc. At each coin toss $X$ falls into the upper third or the lower third of an interval, but never the middle third. en.wikipedia.org/wiki/Cantor_distribution $\endgroup$ – Michael Hardy Nov 28 '15 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.