Suppose that $\{X_1,X_2,\ldots\}$ is a sequence of iid $L^1$-random variables such that $E[X_1]\neq 0$. Define for every $n$, $$ S_n=X_1+\cdots+X_n. $$ Let $N$ be a geometric random variable such that $$ P(N=k) = q^{k-1}p,\quad k=1,2,\ldots, $$ where $q=1-p$ and $p\in(0,1)$. Assume that $N$ and $\{X_1,X_2,\ldots\}$ are all independent. Show that as $p\to0$, $$ {S_N\over E[S_N]} $$ converges in distribution to an exponential distribution with some rate $\lambda$, and identify $\lambda$.

  • $\begingroup$ I have no ideas of addressing this kind of problem. Using characteristic function or in any other ways? $\endgroup$ – Ulton Prinsphield Dec 16 '15 at 9:46
  • $\begingroup$ I think the starting point will be to use Wald's lemma which states that $\mathsf{E}(S_N)=\mathsf{E}(X_1)\mathsf{E}(N)=\frac{\mathsf{E}(X_1)}{p}$ (since $X_1,\ldots, X_n$ are $IID$. Hence \begin{align} \frac{S_N}{\mathsf{E}(S_N)}=\frac{p}{\mathsf{E}(X_1)}S_N \end{align} $\endgroup$ – Shahid M Shah Dec 16 '15 at 9:52
  • $\begingroup$ @ShahidMShah This can also be shown directly with condition expectations (nested). I am not sure how to continue after that, but this seems a good start (conditional expectations should help you "decouple" the $N$ from the $X_k$'s in the expectations). $\endgroup$ – Clement C. Dec 16 '15 at 10:04
  • $\begingroup$ suppose the moment generating function of X is $\psi$, then the mgf of left hand side is $[\psi({pt\over m_1})]^N$, where $m_1=E(X_1)$. So we need to show that $[\psi({pt\over m_1})]^N\to {\lambda\over \lambda-t}$ as $p\to0$. $\endgroup$ – Ulton Prinsphield Dec 16 '15 at 11:44
  • $\begingroup$ The exponent $N$ is a geometric random variable related to $p$, and the mgf function also contains $p$. So the problem remains to be: how could we get the limit of the above formula in this kind of form. I tried to expand the mgf, however I really have no idea how to handle the random variable $N$. $\endgroup$ – Ulton Prinsphield Dec 16 '15 at 11:49

You are already on the right track and I complete the latter part for you. Let $\mu = E[X_1]$ be the common mean and $ \varphi(t) = E[e^{itX_1}]$ be the common characteristic function

As stated above, $$ E[S_N] = E[E[S_N|N]] = E[N\mu] = \frac {\mu} {p} $$

Now consider the characteristic function of $\displaystyle Z = \frac {S_N} {E[S_N]} = \frac {pS_N} {\mu}$:

$$ \begin{align*} \varphi_Z(t) & = E\left[e^{it\frac {pS_N} {\mu}}\right] \\ & = E\left[E\left[e^{it\frac {pS_N} {\mu}}|N\right]\right] \\ & = E\left[E\left[e^{i\frac {tp} {\mu} X_1}\right]^N\right] \\ & = E\left[\varphi\left(\frac {tp} {\mu} \right)^N\right] \\ & = \sum_{k=1}^{+\infty} \varphi\left(\frac {tp} {\mu} \right)^k (1 - p)^{k-1}p \\ & = p\varphi\left(\frac {tp} {\mu} \right) \sum_{k=1}^{+\infty} \left[\varphi\left(\frac {tp} {\mu} \right) (1 - p)\right]^{k-1} \\ & = \frac {\displaystyle p\varphi\left(\frac {tp} {\mu}\right)} {\displaystyle 1 - \varphi\left(\frac {tp} {\mu} \right) (1 - p) } \end{align*}$$

Note the infinite geometric series converges as the characteristic function is bounded by 1. Since $X_1 \in \mathcal{L}^1$, $\varphi$ is differentiable and thus we can evaluate the limit via L'Hopital Rule:

$$ \begin{align*} \lim_{p\to 0} \varphi_Z(t) &= \lim_{p\to 0} \frac {\displaystyle p\varphi\left(\frac {tp} {\mu}\right)} {\displaystyle 1 - \varphi\left(\frac {tp} {\mu} \right) (1 - p) } \\ & = \lim_{p\to 0} \frac {\displaystyle \varphi\left(\frac {tp} {\mu}\right) + p\varphi'\left(\frac {tp} {\mu}\right)\frac {t} {\mu}} {\displaystyle \varphi\left(\frac {tp} {\mu} \right) - (1 - p)\varphi'\left(\frac {tp} {\mu}\right)\frac {t} {\mu} } \\ & = \frac {\varphi(0) + 0} {\displaystyle \varphi(0) - \varphi'(0)\frac {t} {\mu}} \\ & = \frac {1} {\displaystyle 1 - i\mu \frac {t} {\mu}} \\ & = \frac {1} {1 - it} \end{align*}$$

which is the characteristic function of $\text{Exp}(\lambda = 1)$

  • $\begingroup$ impressive! Thank you very much! $\endgroup$ – Ulton Prinsphield Dec 16 '15 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.