# Clarification on convergence in probability

Wikipedia definition

A sequence $X_n$ of random variables converges in probability towards the random variable $X$ if for all $\varepsilon > 0$:

$$\lim_{n\to\infty}\Pr\big(|X_n-X| \geq \varepsilon\big) = 0$$

Formally, pick any $\varepsilon > 0$ and any $\delta > 0$. Let $P_n$ be the probability that $X_n$ is outside the ball of radius $\varepsilon$ centered at $X$. Then for $X_n$ to converge in probability to $X$ there should exist a number $N$ (which will depend on $\varepsilon$ and $\delta$) such that for all $n \geq N$, $P_n < \delta$.

My question

When $X$ is a constant random variable, this definition is easy to understand. But what happens if $X$ is not constant ? Without the expression of the joint distribution $p(X_n, X)$, how can $Pr\big(|X_n - X| \geq \varepsilon\big)$ be computed ?

• Well, the distribution of every $X_n-X$ would suffice (but of course there are many other situations where one can show convergence in probability without knowing this distribution). – Did Apr 2 '16 at 23:00
• the best example is $X_n = X + W_n$ where $W_n$ is some gaussian noise $W_n \sim \mathcal{N}(0,1/n^2)$. here we don't care if $W_n,W_m$ are indepenpent, if you want you can consider the case $X_n = X+ \frac{W_1}{n}$, and clearly the noise $X_n - X$ tends to $0$ when $n \to \infty$ : $X_n$ converges in probability to $X$ – reuns Apr 2 '16 at 23:22
• In this case the probability distribution of the difference is easy to deduce. I added the formulation of the CLT from proofwiki. In that case I can not how to deduce the probability distribution of the difference. – vkubicki Apr 3 '16 at 10:15
• To Did: "there are many other situations where one can show convergence in probability without knowing this distribution". As I understand it, if the joint probability is not provided, there can be multiple distributions for $X_n - X$. So, you might not know it, but how can you know it is unique ? Maybe I do not understand something very fundamental :). – vkubicki Apr 3 '16 at 10:26
• The example you added just muddies things beyond repair: note that the convergence in the CLT is a convergence in distribution which cannot be enhanced to a convergence in probability, even less to an almost sure convergence. Thus, the fact that you ask "how to deduce the expression of the probability distribution of $\frac {S_n - n \mu} {\sqrt {n \sigma^2} } - N \left({0, 1}\right)$" seems to indicate some deep misconception about CLT. (Unrelated: Please use @user to signal a comment to user user.) – Did Apr 3 '16 at 12:27

Regardless of the "technical limitations" to compute it,the definition as it stands is correct; intuitively it means that the probability of finding the a difference between $X_n$ and $X$ bigger than $\epsilon$ goes to $0$.

Now, in most cases to show convergence in probability you don't need the distribution of $X_n - X$; maybe you know that $X_n$ tends to $X$ almost surely (and this implies convergence in probability). Or you can bound the probability $P(|X_n - X| \ge \epsilon) \le a_n \to 0$, which would suffice too.

• Thank you, your answer and Did comments helped a lot. – vkubicki Apr 3 '16 at 15:00

Suppose

• $X$ is uniformly distributed in $(0,1)$;
• $X_1,X_2,X_3,\ldots$ are conditionally independent given $X$;
• The conditional distribution of $X_n$ given $X$, for $n=1,2,3,\ldots$, is $\operatorname{Bernoulli}(X)$, i.e. $$\begin{cases} \Pr(X_n=1\mid X) = X, \\[3pt] \Pr(X_n = 0\mid X) = 1-X. \end{cases}$$

Let $\overline X_n = \dfrac{X_1+\cdots+X_n} n$. Then $\overline X_n$ converges almost surely to $X$, and therefore converges in probability to $X$.