Let $S$ be a finite set, for simplicity assume $S=\{1,2,...,m\}$. Let $f_0$ and $f_1$ be two non-equal probabilities defined on $S$, with $f_0(j)=P_0(X=j)$ and $f_1(j)=P_1(X=j)$, such that $f_0(j)\geq0,f_1(j)\geq0,\sum_{j\in S}f_0(j)=1,\sum_{j\in S}f_1(j)=1$ for every $j\in S$.

Let $X_1,X_2,...,X_n$ be an iid random sample with support $S$. Define:$$Z_n=\prod_{k=1}^n\dfrac{f_1(X_k)}{f_0(X_k)}$$

Show that $Z_n\to0$ almost surely.

I defined $N_n(j)=\sum_{k=1}^n1_{\{X_k=j\}}$ and observed that $Y_n:=\log Z_n=\sum_{j\in S}N_n(j)\log\left(\dfrac{f_1(j)}{f_0(j)}\right)$.

I want to show that $Y_n\to-\infty$ almost surely. I know that $E_0\left(\log\left(\dfrac{f_1(X)}{f_0(X)}\right)\right)<0$ as $f_1\neq f_0$ (by Jensen's Inequality).

Now, $$Y_n=n\sum_{j\in S}\dfrac{N_n(j)}{n}\log\left(\dfrac{f_1(j)}{f_0(j)}\right)$$

Under $f_0$ we have by SLLN, $$\dfrac{N_n(j)}{n}\to f_0(j)$$ almost surely. Hence $$\sum_{j\in S}\dfrac{N_n(j)}{n}\log\left(\dfrac{f_1(j)}{f_0(j)}\right)\to \sum_{j\in S}f_0(j)\log\left(\dfrac{f_1(j)}{f_0(j)}\right)=E_0\left(\log\left(\dfrac{f_1(X)}{f_0(X)}\right)\right)<0$$ and therefore, almost surely,$$\lim_{n\to\infty}Y_n=\lim_{n\to\infty}nE_0\left(\log\left(\dfrac{f_1(X)}{f_0(X)}\right)\right)=-\infty$$ showing that $$Y_n\to-\infty$$ almost surely from which it follows that $$Z_n\to0$$ almost surely.

Is my method correct?

  • $\begingroup$ For a much shorter solution, apply the SLLN to the i.i.d. sequence $(U_n)$ defined by $$U_n=\log f_1(X_n)/f_0(X_n).$$ And please specify the probability measure with respect to which the almost sure statement must be proven. $\endgroup$ – Did Aug 23 '15 at 7:37
  • $\begingroup$ I understand that while saying "a.s. convergence", the prob. space becomes very important. However, this is an exercise problem from a book where the prob. space wasn't defined. $\endgroup$ – Landon Carter Aug 23 '15 at 7:51
  • $\begingroup$ I thought initially to approach the problem in the way you said, @Did. However, I did not understand under which probability $P_1$ or $P_0$, I should apply the SLLN. Because in SLLN, we have to compute expectation, and that can be done under a particular probability only, isn't it? $\endgroup$ – Landon Carter Aug 23 '15 at 7:56
  • $\begingroup$ ?? Naturally the almost sure convergence to zero occurs with respect to one of the probability measures $P_0$ or $P_1$ but not with respect to the other. $\endgroup$ – Did Aug 23 '15 at 8:56
  • $\begingroup$ You may be right. But how can one be so sure? Here nothing is mentioned about either of the probabilities. $\endgroup$ – Landon Carter Aug 23 '15 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.