Suppose that $\{X_1,X_2,\ldots\}$ is a sequence of iid $L^1$-random variables such that $E[X_1]\neq 0$. Define for every $n$, $$ S_n=X_1+\cdots+X_n. $$ Let $N$ be a geometric random variable such that $$ P(N=k) = q^{k-1}p,\quad k=1,2,\ldots, $$ where $q=1-p$ and $p\in(0,1)$. Assume that $N$ and $\{X_1,X_2,\ldots\}$ are all independent. Show that as $p\to0$, $$ {S_N\over E[S_N]} $$ converges in distribution to an exponential distribution with some rate $\lambda$, and identify $\lambda$.

  • $\begingroup$ I have no ideas of addressing this kind of problem. Using characteristic function or in any other ways? $\endgroup$ Dec 16, 2015 at 9:46
  • $\begingroup$ I think the starting point will be to use Wald's lemma which states that $\mathsf{E}(S_N)=\mathsf{E}(X_1)\mathsf{E}(N)=\frac{\mathsf{E}(X_1)}{p}$ (since $X_1,\ldots, X_n$ are $IID$. Hence \begin{align} \frac{S_N}{\mathsf{E}(S_N)}=\frac{p}{\mathsf{E}(X_1)}S_N \end{align} $\endgroup$ Dec 16, 2015 at 9:52
  • $\begingroup$ @ShahidMShah This can also be shown directly with condition expectations (nested). I am not sure how to continue after that, but this seems a good start (conditional expectations should help you "decouple" the $N$ from the $X_k$'s in the expectations). $\endgroup$
    – Clement C.
    Dec 16, 2015 at 10:04
  • $\begingroup$ suppose the moment generating function of X is $\psi$, then the mgf of left hand side is $[\psi({pt\over m_1})]^N$, where $m_1=E(X_1)$. So we need to show that $[\psi({pt\over m_1})]^N\to {\lambda\over \lambda-t}$ as $p\to0$. $\endgroup$ Dec 16, 2015 at 11:44
  • $\begingroup$ The exponent $N$ is a geometric random variable related to $p$, and the mgf function also contains $p$. So the problem remains to be: how could we get the limit of the above formula in this kind of form. I tried to expand the mgf, however I really have no idea how to handle the random variable $N$. $\endgroup$ Dec 16, 2015 at 11:49

1 Answer 1


You are already on the right track and I complete the latter part for you. Let $\mu = E[X_1]$ be the common mean and $ \varphi(t) = E[e^{itX_1}]$ be the common characteristic function

As stated above, $$ E[S_N] = E[E[S_N|N]] = E[N\mu] = \frac {\mu} {p} $$

Now consider the characteristic function of $\displaystyle Z = \frac {S_N} {E[S_N]} = \frac {pS_N} {\mu}$:

$$ \begin{align*} \varphi_Z(t) & = E\left[e^{it\frac {pS_N} {\mu}}\right] \\ & = E\left[E\left[e^{it\frac {pS_N} {\mu}}|N\right]\right] \\ & = E\left[E\left[e^{i\frac {tp} {\mu} X_1}\right]^N\right] \\ & = E\left[\varphi\left(\frac {tp} {\mu} \right)^N\right] \\ & = \sum_{k=1}^{+\infty} \varphi\left(\frac {tp} {\mu} \right)^k (1 - p)^{k-1}p \\ & = p\varphi\left(\frac {tp} {\mu} \right) \sum_{k=1}^{+\infty} \left[\varphi\left(\frac {tp} {\mu} \right) (1 - p)\right]^{k-1} \\ & = \frac {\displaystyle p\varphi\left(\frac {tp} {\mu}\right)} {\displaystyle 1 - \varphi\left(\frac {tp} {\mu} \right) (1 - p) } \end{align*}$$

Note the infinite geometric series converges as the characteristic function is bounded by 1. Since $X_1 \in \mathcal{L}^1$, $\varphi$ is differentiable and thus we can evaluate the limit via L'Hopital Rule:

$$ \begin{align*} \lim_{p\to 0} \varphi_Z(t) &= \lim_{p\to 0} \frac {\displaystyle p\varphi\left(\frac {tp} {\mu}\right)} {\displaystyle 1 - \varphi\left(\frac {tp} {\mu} \right) (1 - p) } \\ & = \lim_{p\to 0} \frac {\displaystyle \varphi\left(\frac {tp} {\mu}\right) + p\varphi'\left(\frac {tp} {\mu}\right)\frac {t} {\mu}} {\displaystyle \varphi\left(\frac {tp} {\mu} \right) - (1 - p)\varphi'\left(\frac {tp} {\mu}\right)\frac {t} {\mu} } \\ & = \frac {\varphi(0) + 0} {\displaystyle \varphi(0) - \varphi'(0)\frac {t} {\mu}} \\ & = \frac {1} {\displaystyle 1 - i\mu \frac {t} {\mu}} \\ & = \frac {1} {1 - it} \end{align*}$$

which is the characteristic function of $\text{Exp}(\lambda = 1)$

  • $\begingroup$ impressive! Thank you very much! $\endgroup$ Dec 16, 2015 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.