My textbook has this theorem:

Suppose that $X_1,X_2,\cdots$ converges in probability to a random variable $X$ and that $h$ is a continuous function. Then $h(X_1), h(X_2),\cdots$ converges in probability to $h(X)$

Failed to come up with a proof immediately, I looked at the proof provided in the solution:

If $h$ is continuous, given $\epsilon > 0$ there exits $\delta>0$ such that $|h(x_n)−h(x)| < \epsilon$  for $|x_n−x| < \delta$. Since $X_1,\cdots, X_n$ converges in probability to the random variable $X$, then $\lim_{n\to\infty} P(|X_n − X| < \delta) = 1$. Thus $\lim_{n\to\infty} P(|h(X_n) − h(X)| < \epsilon) = 1$.

The part follows "since" does not really make sense to me. I think the subtly lies in the fact that $X_n$ is a random variable (that is, a function defined on some sigma-algebra $\Omega$). So for a given $\omega \in \Omega$, for a given $n$, and for a given $\epsilon >0$, we indeed can find $\delta$ such that $$|X_n( \omega) - X (\omega)| < \delta \text{ implies that } |h(X_n( \omega) - h(X (\omega))|<\epsilon$$

, but the choice of such $\delta$ seems to depend on $\omega$ and $n$? My expectation was that, in the end, we want something like $$\{\omega\in \Omega: |X_n(\omega) - X(\omega)|<\delta\} \subset \{ \omega\in\Omega: |h(X_n(\omega) - h(X(\omega))| < \epsilon\}$$

  • $\begingroup$ What were the exact hypotheses on $h$. In particular, do they assume $h$ is uniformly continuous? $\endgroup$ – kimchi lover Sep 24 '17 at 2:11
  • $\begingroup$ @kimchilover Well, it doesn't say uniformly continuous in the theorem. $\endgroup$ – 3x89g2 Sep 24 '17 at 2:12
  • 1
    $\begingroup$ Look at the wikipedia article on continuous mapping theorem. It's clear that your textbook's "proof" is an abridgment of the argument given there. I'm not 100% sure of the details of the wikipedia argument, but it is a step in the right direction. $\endgroup$ – kimchi lover Sep 24 '17 at 2:32
  • 1
    $\begingroup$ The claimed proof is wrong because the statement "given $\epsilon > 0$ there exists $\delta>0$ such that $|h(x_n)−h(x)| < \epsilon$ for $|x_n - x| < \delta$" is not generally true for continuous functions; it's the definition of uniformly continuous. But if $h$ were uniformly continuous the proof would be fine. The $\delta$ really does only depend on $\epsilon$ and not on anything else in sight. The choice to call the second point $x_n$ is perhaps part of the confusion; they should have written "$|h(y) - h(x)| < \epsilon$ for $|x-y| < \delta$." $\endgroup$ – Nate Eldredge Sep 24 '17 at 5:15
  • 1
    $\begingroup$ @Misakov: Yes, correct. If $h$ is merely continuous then $\delta$ also depends on $x$. $\endgroup$ – Nate Eldredge Sep 24 '17 at 15:00

The OP is right: the "proof" in the book is incomplete. It assumes $h$ is uniformly continuous, that is, that for every $\epsilon>0$ there is a single $\delta>0$ such that for all $x$ the usual good stuff happens. The proof sketched in the wikipedia article is more serious, if hard to follow. My library does not have a copy of the book the wikipedia's proof seems taken from, so I came up with a separate argument, which is (I think) a bit more conceptual than the wikipedia's argument, or at least relies on standard facts in a more self-evident way.

These facts are: in a separable metric space $(S,d)$, convergence in probability is metrizable by the metric $d_P(X,Y)=E\min(1,d(X,Y))$, and in a separable metric space, convergence in probability $X_n\xrightarrow P X$ implies convergence in distribution $X_n\xrightarrow D X$. See the wikipedia article or the discussion in Billingsley's Convergence of Probability Measures, (first, 1968 edition) p.26.

Here is a sketch of how to use them to prove the result. We have $X_n\xrightarrow{P}X,$ where the rvs are $S$-valued, and $h:S\to S'$ is continuous. Then we also have $(X_n,X)\xrightarrow{P}(X,X)$ and $(X_n,X)\xrightarrow{D}(X,X)$. (The first follows from use of $d_P$; the second is standard.) Define $g:S\times S\to \mathbb R$ by $g(u,v) = \max(1,d(h(u),h(v)))$, where $d(\cdot,\cdot)$ is the metric on $S'$. It is clear that $g$ is continuous and bounded, so $(X_n,X)\xrightarrow{D}(X,X)$ implies $\lim_{n\to\infty} Eg(X_n,X) = Eg(X,X)$. But $Eg(X,X) = 0$, so we know $\lim_{n>0} d_P(Y_n,Y)=0,$ where $Y_n=h(X_n)$ and $Y=h(X).$ This implies $Y_n\xrightarrow P Y$.

ADDED NEXT DAY: Grimmett's textbook has a clean and simple proof. We want to show that for arbitrary $\delta>0$ we have $P(|Y_n-Y|>\delta)\to0$. It suffices to show that for each $\eta>0$ for all $n$ sufficiently large, $P(|Y_n-Y|>\delta)<\eta$. So pick $K$ so large that $P(|X|>K)<\eta/2$, and observe that the function $h$ is uniformly continuous on $[-K,K]$, and hence there exists an $\epsilon>0$ such that $|u|\le K$ and $|u-v|\le\epsilon$ together imply $|h(u)-h(v)|<\delta$. Thus, $P(|Y_n-Y|>\delta)<P(|X|>K) + P(|X_n-X|>\epsilon)<\eta/2 + P(|X_n-X|>\epsilon)<\eta$ for all $n$ sufficiently large.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.