The definition of a Cumulative Distribution Function $(CDF)$ says that $$P(X \le x) = F(x)$$

This is all good.

Then my text book gives the following theorem without proof: $$P(X \lt x) = F(x-)$$

The book says that the proof is easy, but with my rusted calculus skills, I have trouble to even intuitively understand this theorem.

I tried to sketch a proof, to improve my understanding: $$F(x-) = \lim_{n \to \infty} F(x - \frac{1}{n}) = \lim_{n \to \infty} P(X < x - \frac{1}{n}) = P(X < x).$$

I assume we can't say $P(X \le x - \frac{1}{n})$, because then $X(\omega) \ge x$, for some $\omega \in \Omega$.

Is this the correct reasoning?

  • $\begingroup$ If you have $F(x) = P(X\leq x)$ in the definition, you cannot change it to $F(x) = P(X < x)$. So you should have "$\leq$" instead of "<" in $P(X < x-\frac{1}{n})$. $\endgroup$ – Djaian Nov 1 '10 at 20:23

The intuition is that the CDF is just "adding probabilities up", a little bit at a time. Remember, the integral is a sum; that's why $\int$ is an elongated 's'.

So think of your expression $F(x-\frac 1 n)$ as "adding a little bit more"; as $n$ gets larger, you added in the difference $$ \int_{-\infty}^{x-\frac 1 {n+1}} f(x) - \int_{-\infty}^{x-\frac 1 n} f(x) = \int_{x-\frac 1 n}^{x-\frac 1 {n+1}} f(x). $$ Since $f(x) \geq 0$ for all $x$, this difference is also $\geq 0$. The limit is then the sum of the area under the curve $f(x)$ between $x-1/n$ and $x-1/(n+1)$: $$ \sum_{n=1}^\infty \int_{x-\frac 1 n}^{x-\frac 1 {n+1}} f(x). $$

Now, if $X$ is a continuous random variable, this sum will converge to $F(x)$. However, if $X$ has "discrete pieces" things can go wrong and the CDF can jump: see the third picture in this diagram. In general, however, the sum will converge to $\lim_{b\to x^-} F(b)$.

  • $\begingroup$ Hi, Josh! I have to keep saying hi so that this comment is long enough. $\endgroup$ – Michael Lugo Nov 1 '10 at 23:11
  • $\begingroup$ Thanks Josh. My book doesn't define CDF in terms of the Probability Density Function. At least not yet. So at this point I'm resorting to Djaian's answer, because I'm not very familiar with the properties of PDF. $\endgroup$ – Rutherford Nov 2 '10 at 15:49
  • $\begingroup$ @Michael: Hey! How's Berkeley? $\endgroup$ – Josh Guffin Nov 2 '10 at 22:06
  • $\begingroup$ @Rutherford: the PDF is just the derivative of the CDF - it's usually defined the other way, but they're equivalent by the fundamental theorem of calculus. Maybe it would help to note that you can, with a little bit of manipulation, turn the difference $F(x) - F(x^-) = \lim_{n\to\infty} [F(x) - F(x- 1/n)]$ into a derivative, which will vanish unless $P(X = x) \ne 0$. The nonvanishing of this probability is equivalent to saying that $X$ is not a "purely continuous" random variable. $\endgroup$ – Josh Guffin Nov 2 '10 at 22:19
  • $\begingroup$ I guess that I could write the derivatives in at least two ways: $lim_{n \to \infty} [\frac{F(x^- + 1/n) - F(x^-)}{1/n}]$ or $lim_{n \to \infty} [\frac{F(x) - F(x - 1/n)}{1/n}]$. But how do they vanish when $P(X=x) = 0$? $\endgroup$ – Rutherford Nov 3 '10 at 8:41

For $x$ fixed, define events $A_n$ by $A_1 = { X \leq x-1 }$ and, for $n \geq 2$, $A_n = { x-\frac{1}{{n - 1}} < X \leq x-\frac{1}{n} }$. Since the disjoint union $A_1 \cup A_2 \cup \ldots$ gives the event ${ X < x }$, we have ${\rm P}(X < x)= {\rm P}(A_1 \cup A_2 \cup \ldots) = \sum\nolimits_{k = 1}^\infty {{\rm P}(A_k )}$. Noting that $\sum\nolimits_{k = 1}^n {{\rm P}(A_k )} = {\rm P}(X \leq x - \frac{1}{n})$, we thus obtain the desired result: ${\rm P}(X < x) = \lim _{n \to \infty } {\rm P}(X \le x - \frac{1}{n}) = \lim _{n \to \infty } F(x - \frac{1}{n}) = F(x - )$.

  • $\begingroup$ Thanks Shai. That proof I can understand, but a proper Latex formatting would be nice. I'd still like to see if I can modify Djaian's suggestions into a working proof. $\endgroup$ – Rutherford Nov 2 '10 at 16:59
  • $\begingroup$ Apparently, the key point in the problem is to show that ${\rm P}(X < x) = \lim _{n \to \infty } {\rm P}(X \le x - \frac{1}{n})$. $\endgroup$ – Shai Covo Nov 2 '10 at 17:59

EDIT: I rewrite my answer, since it appears it was not clear enough. I will try to go in more details even though I think this is not really interesting.

We define $F(x) := P(X \leq x)$ and we want to prove $F(x-) = P(X < x)$.

Consider a sequence $(a_n)$ such that $\lim_{n \rightarrow \infty} a_n = x$, $a_n < x$ for all $n$. For the sake of simplicity, let's admit $a_n$ is increasing ($a_{n+1} \geq a_n$).

Now, let's consider the sequence $(b_n)$ given by $b_n := F(a_n)$. By definition of $F$, we have $b_n = P(X \leq a_n)$. Thus $(b_n)$ is a sequence in [0,1]. Since $b_{n+1} = P(X \leq a_{n+1}) \geq P(X \leq a_n) = b_n$, we have that the sequence $b_n$ is also increasing. Hence, it converges. Now let's calculate the limit (let's call it $b$).

We know that $P(X \leq a_n) \leq P(X < x)$, so $b_n \leq P(X < x)$ for all $n$. Thus this should remain true for the limit $b$. You might argue that you are not sure why this should remain true for the limit and why we could not have $b = P(X \leq x)$. Here comes the $\varepsilon$ argument. If $P(X \leq x) = P(X < x)$ then there is no problem, so suppose $P(X \leq x) \ne P(X < x)$. In that case, let $\varepsilon := \frac{1}{2}\left(P(X \leq x) - P(X < x)\right)$.

By definition of the limit, you should find infinite members of the sequence in $[b- \varepsilon, b + \varepsilon]$. But this can't be, because all the $b_n$ are smaller than $P(X < x)$ and thus at distance at least 2 $\varepsilon$ from the limit. This proves that if $P(X \leq x) \ne P(X < x)$, then $b \ne P(X \leq x)$.

Ok, so we know $b \leq P(X < x)$. We still need to prove there is an equality here. I will not go into further technical details here, the intuition should tell you that since $a_n \rightarrow x$, you can come as near to $P(X < x)$ as you want with the sequence $b_n$ (consider $\varepsilon = \frac{P(X < x) -b}{2}$ and see what this implies bla-bla-bla).

In the beginning I said we could consider $a_n$ to be increasing. Suppose it is not. Then you can construct a new sequence $c_n$, such that $\lim c_n = x$, $c_n \leq a_n$ for all $n$ and $c_n$ is increasing. Then you compare the sequence $d_n := F(c_n)$ with the sequence $b_n$ and bla-bla-bla. This really becomes technical, but it works. So $F(x-) = P(X < x)$.

by the way, sorry if I can't write my sequences properly, I can't get { } in math mode, even with 1 backslash, 2 backslaches or 3 backslaches before the {. This really annoys me.

  • $\begingroup$ I like this approach, but the proof still eludes me. I can't figure out how to make use of the epsilon, and why did you choose $\epsilon = \frac{p}{2}$? $\endgroup$ – Rutherford Nov 2 '10 at 15:53
  • $\begingroup$ Thanks Djaian. For { and } in math mode, try \lbrace and \rbrace. I'm not finished reading your edits, but I'd like to add some clarifications: $b_n \not\in [b - \epsilon, b + \epsilon]$, because $2\epsilon = P(X \le x) - P(X < x)$ and $\lim_{n \to \infty} b_n - b \ge 2\epsilon $. These propositions weren't immediately clear to me, so probably they help someone else too. $\endgroup$ – Rutherford Nov 4 '10 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.