How to prove that geometric distributions converge to an exponential distribution?

To solve this, I am trying to define an indexing $n$/$m$ and to send $m$ to infinity, but I get zero, not some relevant distribution. What is the technique or approach one must use here?


Recall pmf of the geometric distribution: $\mathbb{P}(X = k) = p (1-p)^k$ for $k \geq 0$.

The geometric distribution has the interpretation of the number of failures in a sequence of Bernoulli trials until the first success.

Consider a regime when the probability of success is very small, such that $n p = \lambda$, and consider $x = \frac{k}{n}$. Then, in the large $n$ limit: $$ 1 = \sum_{k=0}^\infty \mathbb{P}(X = k) = \sum_{k=0}^\infty \lambda \left(\left(1 - \frac{\lambda}{n} \right)^{n \cdot k/n} \frac{1}{n} \right) \stackrel{n \to \infty}\rightarrow \int_0^\infty \lambda \mathrm{e}^{-\lambda x} \mathrm{d} x $$

Alternatively, you could look at the moment generating function for the geometric distribution: $$ \mathcal{M}(p, t) = \frac{p}{1-\mathrm{e}^t (1-p)} $$ To recover the mgf of the exponential distribution consider the limit: $$ \lim_{n \to \infty} \mathcal{M}\left( \frac{\lambda}{n}, \frac{t}{n} \right) = \lim_{n \to \infty} \frac{\lambda}{n - \mathrm{e}^{t/n}\left(n - \lambda\right)} = \lim_{n \to \infty} \frac{\lambda}{n \left(1 - \mathrm{e}^{t/n}\right) + \lambda \mathrm{e}^{t/n} } = \frac{\lambda}{\lambda-t} $$ which is the mgf of the exponential distribution with rate $\lambda$.

  • $\begingroup$ there needs to be some change of variables in the sum and we're set. I guess the technique in finding continuous analogues is to sum over all k's instead of just trying to do something with one k. $\endgroup$ – Ramamurthy Shankar Dec 21 '11 at 4:52
  • $\begingroup$ Why do you take the limit as t goes to zero? $\endgroup$ – SSD Jul 28 '18 at 23:09
  • $\begingroup$ @SSD I take a limit of $n \to \infty$ $\endgroup$ – Sasha Jul 29 '18 at 0:58
  • $\begingroup$ Well, I guess I was wondering how an n got into the parameter of the moment generating function? I was under the impression that all that was for was taking derivatives. $\endgroup$ – SSD Jul 29 '18 at 1:12

It may be useful to construct the relevant distributions the other way round, that is, to start from a single exponential random variable $X_\lambda$ with density $\lambda\mathrm e^{-\lambda x}$ on $x\geqslant0$, for a given positive $\lambda$, and to consider the integer part $Y_a$ of $X_\lambda/a$, for every positive $a$.

Then, $[Y_a=n]=[na\leqslant X_\lambda\lt (n+1)a]$ hence $\mathrm P(Y_a=n)=\mathrm e^{-\lambda na}-\mathrm e^{-\lambda (n+1)a}$ for every nonnegative integer $n$, that is, $$\mathrm P(Y_a=n)=(p_a)^n(1-p_a),$$ with $$ p_a=\mathrm e^{-\lambda a}. $$ This proves that each $Y_a$ is geometric with parameter $p_a$. Note that, for a given $\lambda$, every parameter $p$ in $(0,1)$ is realized as $p=p_a$ for some $a$, hence, starting from a unique random variable $X_\lambda$, this construction yields a whole family of geometric random variables $(Z_p)_{p\in(0,1)}$ such that each $Z_p$ is geometric with parameter $p$ and such that $Z_p\leqslant Z_q$ with full probability, for every $p\geqslant q$ in $(0,1)$: simply define each $Z_p$ by $Z_p=Y_{\alpha(p)}$, with $$ \alpha(p)=-\log(p)/\lambda. $$ Coming back to your question, $X_\lambda\leqslant aY_a\lt X_\lambda+a$ with full probability, hence $aY_a\to X_\lambda$ almost surely when $a\to0$, in particular $\alpha(p)Z_p\to X_\lambda$ when $p\to1$. Thus, $-\log(p)Z_p$ converges almost surely (hence in distribution) to the standard exponential random variable $X_1=\lambda X_\lambda$.

Since $-\log(p)\sim1-p$ when $p\to1$, this shows that $(1-p)Z_p$ converges almost surely (hence also in distribution) to $X_1$. Now, the convergence in distribution of some random variables does not depend on their realization hence, by the remarks above, for every family $(T_p)_{p\in(0,1)}$ of random variables such that each $T_p$ is geometrically distributed with parameter $p$, the random variables $(1-p)T_p$ converge in distribution to the standard exponential distribution when $p\to1$.

Note Such a simultaneous and almost sure construction of geometric random variables $Z_p$, for every $p$ in $(0,1)$, from a single (exponentially distributed) random variable $X_\lambda$ is called a coupling. For more about this powerful idea, see the WP page on coupling in probability.


The waiting time $T$ until the first success in a sequence of independent Bernoulli trials with probability $p$ of success in each one has a geometric distribution with parameter $p$: its probability mass function is $P(x) = p (1-p)^{x-1}$ and cumulative distribution function $F(x) = 1 - (1-p)^x$ for positive integers $x$ (note that some authors use a different convention where the random variable is $T-1$ rather than $T$, but that won't make a difference in the limit). The scaled version $p T$ converges in distribution as $p \to 0+$ to an exponential random variable with rate $1$, as for $x \ge 0$ $$ P(p T \le x) = F(x/p) = 1 - (1-p)^{\lfloor x/p\rfloor} \to 1 - e^{-x} $$

  • $\begingroup$ Can you show more details about how $$(1-p)^{\lfloor x/p\rfloor} \to e^{-x}$$. $\endgroup$ – 81235 Jan 19 '15 at 15:08
  • 4
    $\begingroup$ Since $0 \le x/p - \lfloor x/p \rfloor \le 1$, $(1-p)^{\lfloor x/p \rfloor - x/p} \to 1$, while $\ln \left((1-p)^{x/p}\right) = x \dfrac{\ln(1-p)}{p} \to -x$ by definition of derivative of $\ln(x)$ at $x=1$. $\endgroup$ – Robert Israel Jan 19 '15 at 18:04

I think we can just expand 1-p in Taylor series for small p we get the exponential function.:) 1-p=exp(-p) for very small p (around p=0)such that np=constant. so we find:

p(x)=(1-p)^x.p is becoming p exp(-px) for a continuous random variable x, this in part has the continuous exponential distribution function:)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.