# How to prove that geometric distributions converge to an exponential distribution?

How to prove that geometric distributions converge to an exponential distribution?

To solve this, I am trying to define an indexing $n$/$m$ and to send $m$ to infinity, but I get zero, not some relevant distribution. What is the technique or approach one must use here?

Recall pmf of the geometric distribution: $\mathbb{P}(X = k) = p (1-p)^k$ for $k \geq 0$.

The geometric distribution has the interpretation of the number of failures in a sequence of Bernoulli trials until the first success.

Consider a regime when the probability of success is very small, such that $n p = \lambda$, and consider $x = \frac{k}{n}$. Then, in the large $n$ limit: $$1 = \sum_{k=0}^\infty \mathbb{P}(X = k) = \sum_{k=0}^\infty \lambda \left(\left(1 - \frac{\lambda}{n} \right)^{n \cdot k/n} \frac{1}{n} \right) \stackrel{n \to \infty}\rightarrow \int_0^\infty \lambda \mathrm{e}^{-\lambda x} \mathrm{d} x$$

Alternatively, you could look at the moment generating function for the geometric distribution: $$\mathcal{M}(p, t) = \frac{p}{1-\mathrm{e}^t (1-p)}$$ To recover the mgf of the exponential distribution consider the limit: $$\lim_{n \to \infty} \mathcal{M}\left( \frac{\lambda}{n}, \frac{t}{n} \right) = \lim_{n \to \infty} \frac{\lambda}{n - \mathrm{e}^{t/n}\left(n - \lambda\right)} = \lim_{n \to \infty} \frac{\lambda}{n \left(1 - \mathrm{e}^{t/n}\right) + \lambda \mathrm{e}^{t/n} } = \frac{\lambda}{\lambda-t}$$ which is the mgf of the exponential distribution with rate $\lambda$.

• there needs to be some change of variables in the sum and we're set. I guess the technique in finding continuous analogues is to sum over all k's instead of just trying to do something with one k. – Ramamurthy Shankar Dec 21 '11 at 4:52
• Why do you take the limit as t goes to zero? – SSD Jul 28 '18 at 23:09
• @SSD I take a limit of $n \to \infty$ – Sasha Jul 29 '18 at 0:58
• Well, I guess I was wondering how an n got into the parameter of the moment generating function? I was under the impression that all that was for was taking derivatives. – SSD Jul 29 '18 at 1:12

It may be useful to construct the relevant distributions the other way round, that is, to start from a single exponential random variable $X_\lambda$ with density $\lambda\mathrm e^{-\lambda x}$ on $x\geqslant0$, for a given positive $\lambda$, and to consider the integer part $Y_a$ of $X_\lambda/a$, for every positive $a$.

Then, $[Y_a=n]=[na\leqslant X_\lambda\lt (n+1)a]$ hence $\mathrm P(Y_a=n)=\mathrm e^{-\lambda na}-\mathrm e^{-\lambda (n+1)a}$ for every nonnegative integer $n$, that is, $$\mathrm P(Y_a=n)=(p_a)^n(1-p_a),$$ with $$p_a=\mathrm e^{-\lambda a}.$$ This proves that each $Y_a$ is geometric with parameter $p_a$. Note that, for a given $\lambda$, every parameter $p$ in $(0,1)$ is realized as $p=p_a$ for some $a$, hence, starting from a unique random variable $X_\lambda$, this construction yields a whole family of geometric random variables $(Z_p)_{p\in(0,1)}$ such that each $Z_p$ is geometric with parameter $p$ and such that $Z_p\leqslant Z_q$ with full probability, for every $p\geqslant q$ in $(0,1)$: simply define each $Z_p$ by $Z_p=Y_{\alpha(p)}$, with $$\alpha(p)=-\log(p)/\lambda.$$ Coming back to your question, $X_\lambda\leqslant aY_a\lt X_\lambda+a$ with full probability, hence $aY_a\to X_\lambda$ almost surely when $a\to0$, in particular $\alpha(p)Z_p\to X_\lambda$ when $p\to1$. Thus, $-\log(p)Z_p$ converges almost surely (hence in distribution) to the standard exponential random variable $X_1=\lambda X_\lambda$.

Since $-\log(p)\sim1-p$ when $p\to1$, this shows that $(1-p)Z_p$ converges almost surely (hence also in distribution) to $X_1$. Now, the convergence in distribution of some random variables does not depend on their realization hence, by the remarks above, for every family $(T_p)_{p\in(0,1)}$ of random variables such that each $T_p$ is geometrically distributed with parameter $p$, the random variables $(1-p)T_p$ converge in distribution to the standard exponential distribution when $p\to1$.

Note Such a simultaneous and almost sure construction of geometric random variables $Z_p$, for every $p$ in $(0,1)$, from a single (exponentially distributed) random variable $X_\lambda$ is called a coupling. For more about this powerful idea, see the WP page on coupling in probability.

The waiting time $T$ until the first success in a sequence of independent Bernoulli trials with probability $p$ of success in each one has a geometric distribution with parameter $p$: its probability mass function is $P(x) = p (1-p)^{x-1}$ and cumulative distribution function $F(x) = 1 - (1-p)^x$ for positive integers $x$ (note that some authors use a different convention where the random variable is $T-1$ rather than $T$, but that won't make a difference in the limit). The scaled version $p T$ converges in distribution as $p \to 0+$ to an exponential random variable with rate $1$, as for $x \ge 0$ $$P(p T \le x) = F(x/p) = 1 - (1-p)^{\lfloor x/p\rfloor} \to 1 - e^{-x}$$

• Can you show more details about how $$(1-p)^{\lfloor x/p\rfloor} \to e^{-x}$$. – 81235 Jan 19 '15 at 15:08
• Since $0 \le x/p - \lfloor x/p \rfloor \le 1$, $(1-p)^{\lfloor x/p \rfloor - x/p} \to 1$, while $\ln \left((1-p)^{x/p}\right) = x \dfrac{\ln(1-p)}{p} \to -x$ by definition of derivative of $\ln(x)$ at $x=1$. – Robert Israel Jan 19 '15 at 18:04

I think we can just expand 1-p in Taylor series for small p we get the exponential function.:) 1-p=exp(-p) for very small p (around p=0)such that np=constant. so we find:

p(x)=(1-p)^x.p is becoming p exp(-px) for a continuous random variable x, this in part has the continuous exponential distribution function:)