How does a geometric distribution converge to an exponential distribution?

I am trying to define an indexing $n$/$m$ and I am sending $m$ to infinity, but I get zero...not some relevant distribution. What is the technique or approach one must use here?!

-

The waiting time $T$ until the first success in a sequence of independent Bernoulli trials with probability $p$ of success in each one has a geometric distribution with parameter $p$: its probability mass function is $P(x) = p (1-p)^{x-1}$ and cumulative distribution function $F(x) = 1 - (1-p)^x$ for positive integers $x$ (note that some authors use a different convention where the random variable is $T-1$ rather than $T$, but that won't make a difference in the limit). The scaled version $p T$ converges in distribution as $p \to 0+$ to an exponential random variable with rate $1$, as for $x \ge 0$ $$P(p T \le x) = F(x/p) = 1 - (1-p)^{\lfloor x/p\rfloor} \to 1 - e^{-x}$$

-
Can you show more details about how $$(1-p)^{\lfloor x/p\rfloor} \to e^{-x}$$. – 81235 Jan 19 '15 at 15:08
Since $0 \le x/p - \lfloor x/p \rfloor \le 1$, $(1-p)^{\lfloor x/p \rfloor - x/p} \to 1$, while $\ln \left((1-p)^{x/p}\right) = x \dfrac{\ln(1-p)}{p} \to -x$ by definition of derivative of $\ln(x)$ at $x=1$. – Robert Israel Jan 19 '15 at 18:04

It may be useful to consider things the other way round, that is, to start from an exponential random variable $X_\lambda$ with density $\lambda\mathrm e^{-\lambda x}$ on $x\geqslant0$, for a given positive $\lambda$, and, for every positive $a$, to consider the integer part $Y_a$ of $X_\lambda/a$.

Then $[Y_a=n]=[na\leqslant X_\lambda\lt (n+1)a]$ hence $\mathrm P(Y_a=n)=\mathrm e^{-\lambda na}-\mathrm e^{-\lambda (n+1)a}$. One sees that $\mathrm P(Y_a=n)=p_a^n(1-p_a)$ with $$p_a=\mathrm e^{-\lambda a}.$$ This proves that each $Y_a$ is geometric with parameter $p_a$. Note that, for a given $\lambda$, every parameter $p$ in $(0,1)$ is realized as $p=p_a$ for some $a$, hence this construction yields a whole family of geometric random variables $(Z_p)_p$ such that each $Z_p$ is geometric with parameter $p$ and $Z_p\leqslant Z_q$ with full probability, for every $p\geqslant q$: simply define each $Z_p$ by $Z_p=Y_{\alpha(p)}$, with $$\alpha(p)=\log(1/p)/\lambda.$$ Coming back to your question, $X_\lambda\leqslant aY_a\lt X_\lambda+a$ with full probability, hence $aY_a\to X_\lambda$ almost surely when $a\to0$, in particular $\alpha(p)Z_p\to X_\lambda$ when $p\to1$. Thus, $\log(1/p)Z_p$ converges almost surely (hence in distribution) to the standard exponential random variable $X_1=X_\lambda/\lambda$.

Since $\log(1/p)\sim1-p$ when $p\to1$, this shows that $(1-p)Z_p$ converges almost surely (hence also in distribution) to $X_1$. Now, the convergence in distribution of some random variables does not depend on their realization hence, by the remarks above, for every family $(T_p)_{0\lt p\lt1}$ of random variables such that each $T_p$ is geometrically distributed with parameter $p$, the random variables $(1-p)T_p$ converge in distribution to a standard exponential distribution when $p\to1$.

Note Such a simultaneous and almost sure construction of some geometric random variable $Z_p$ for every $p$ in $(0,1)$ from a single (exponentially distributed) random variable $X_\lambda$ is called a coupling. For more about this powerful idea, see here.

-

Recall pmf of the geometric distribution: $\mathbb{P}(X = k) = p (1-p)^k$ for $k \geq 0$.

The geometric distribution has the interpretation of the number of failures in a sequence of Bernoulli trials until the first success.

Consider a regime when the probability of success is very small, such that $n p = \lambda$, and consider $x = \frac{k}{n}$. Then, in the large $n$ limit: $$1 = \sum_{k=0}^\infty \mathbb{P}(X = k) = \sum_{k=0}^\infty \lambda \left(\left(1 - \frac{\lambda}{n} \right)^{n \cdot k/n} \frac{1}{n} \right) \stackrel{n \to \infty}\rightarrow \int_0^\infty \lambda \mathrm{e}^{-\lambda x} \mathrm{d} x$$

Alternatively, you could look at the moment generating function for the geometric distribution: $$\mathcal{M}(p, t) = \frac{p}{1-\mathrm{e}^t (1-p)}$$ To recover the mgf of the exponential distribution consider the limit: $$\lim_{n \to \infty} \mathcal{M}\left( \frac{\lambda}{n}, \frac{t}{n} \right) = \lim_{n \to \infty} \frac{\lambda}{n - \mathrm{e}^{t/n}\left(n - \lambda\right)} = \lim_{n \to \infty} \frac{\lambda}{n \left(1 - \mathrm{e}^{t/n}\right) + \lambda \mathrm{e}^{t/n} } = \frac{\lambda}{\lambda-t}$$ which is the mgf of the exponential distribution with rate $\lambda$.

-
there needs to be some change of variables in the sum and we're set. I guess the technique in finding continuous analogues is to sum over all k's instead of just trying to do something with one k. – Ramamurthy Shankar Dec 21 '11 at 4:52