So I know that a cdf F is defined via F(a) = Pr[X ≤ a]. How do I show that the cdf F of a random variable X contains exactly the same information as the function defined via G(a) = Pr[X = a], by expressing F in terms of G and expressing G in terms of F.

-- Compute and plot the cdf for (i) X ∼ Geom(p), (ii) X ∼ Exp(λ ).

-- Identify two key properties that a cdf of any r.v. has to satisfy.

  • $\begingroup$ In general, F cannot be expressed in terms of G. In your example (ii), G=0 identically and there is no way to guess F from that information. $\endgroup$ – Did Nov 17 '11 at 9:43
  • $\begingroup$ I changed the subject line. There is no such thing as a cumulative density function. The word "cumulative" contradicts the word "density". $\endgroup$ – Michael Hardy Nov 17 '11 at 16:32

For the starting period you should develop a clear distinction between discrete random variables and continuous ones:

  1. Discrete random variable $X$ takes only finite (or sometimes countable) number of finite real values $\{x_i\}_{i\in I}$ where $I$ is either finite $I = 1,2,...,n$ or countable $I = \mathbb N$. So, for each value we can assign a probability that $X$ will take this single value: $$ p_X(x_i) = \mathsf P\{X = x_i\} $$ and $p(x_i)$ is called probability mass function (PMF). PMF uniquely characterizes distribution of a discrete random variable, but there is another way to do it: through cumulative distribution function (CDF). For any real $x$ we define $$ F_X(x) := \mathsf P\{X\leq x\} = \sum\limits_{x_i\leq x}p_X(x_i) $$ where the sum is taken over all $i$ such that $x_i\leq x$. You can easily see two (or three :) ) properties of $F_X$: first, $F_X(-\infty) = \lim\limits_{x\to-\infty}F_X(x) = 0$ and $F_X(\infty) = \lim\limits_{x\to\infty}F_X(x)= 1$ and second, $F_X$ is monotonically non-increasing because if $x'\leq x''$ then $\{X\leq x'\}\subseteq \{X\leq x''\}$ and by monotonicity of $\mathsf P$ we get $$ F_{X}(x') = \mathsf P\{X\leq x'\}\leq\mathsf P\{X\leq x''\} = F_X(x''). $$ CDF characterizes PMF and hence the whole distribution of $X$ uniquely. We has already shown how to construct CDF from PMF through the sum, let us show the opposite construction: $$ p_X(x_i) = \mathsf P\{X\leq x_i\} - \mathsf P\{X<x_i\} = F_{X}(x_i)-\lim\limits_{x\to x_i-0}F(x). $$ It may seem unnecessary complication to use CDF, but in contrast to PMF it can be easily generalized.

  2. Continuous real-valued random variable $Y$ takes uncountably many values and due to this complication we cannot assign probability for each single value in an informative way: indeed for any $y\in \mathbb R$ it holds that $$ \mathsf P\{Y=y\} = 0 $$ so there is no useful information about $Y$ at all from the equality (as Didier has already told you). That's why instead of using PMF which would say nothing to us, the use of CDF still makes sense. As an analogue of PMF one talks about probability density function (PDF) which is a derivative of CDF: $$ f_Y(y):=\frac{\mathrm d}{\mathrm dy}F_Y(y) $$ and hence $$ F_Y(y) = \int\limits_{-\infty}^y f_Y(t)\mathrm dt. $$ As a minor note, PDF is not a probability of something. In contrast to CDF and PMF it can take value larger than $1$. PDF is rather a likelihood that random variable will take a specific value, not a probability. As an example, let us consider the case of exponential distribution, i.e. $$ f_Y(y) = \lambda \mathrm e^{-\lambda y}\cdot 1_{y\geq 0} $$ so for any $y>0$ we have $$ F_Y(y) = \int\limits_{-\infty}^y f_Y(t)\mathrm dt = \int\limits_{0}^y \lambda \mathrm e^{-\lambda t}\mathrm dt = -\mathrm e^{-\lambda t}|_{0}^y = 1-\mathrm e^{-\lambda y}. $$

I hope it's now more clear to you, so you just should consider the case when $y\geq 0$ and the case of geometrical distribution. Also, you may be interested in proving the kind of continuity which CDF performs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.