Wikipedia definition

A sequence $X_n$ of random variables converges in probability towards the random variable $X$ if for all $\varepsilon > 0$:

\begin{equation} \lim_{n\to\infty}\Pr\big(|X_n-X| \geq \varepsilon\big) = 0 \end{equation}

Formally, pick any $\varepsilon > 0$ and any $\delta > 0$. Let $P_n$ be the probability that $X_n$ is outside the ball of radius $\varepsilon$ centered at $X$. Then for $X_n$ to converge in probability to $X$ there should exist a number $N$ (which will depend on $\varepsilon$ and $\delta$) such that for all $n \geq N$, $P_n < \delta$.

My question

When $X$ is a constant random variable, this definition is easy to understand. But what happens if $X$ is not constant ? Without the expression of the joint distribution $p(X_n, X)$, how can $Pr\big(|X_n - X| \geq \varepsilon\big)$ be computed ?

  • $\begingroup$ Well, the distribution of every $X_n-X$ would suffice (but of course there are many other situations where one can show convergence in probability without knowing this distribution). $\endgroup$ – Did Apr 2 '16 at 23:00
  • $\begingroup$ the best example is $X_n = X + W_n$ where $W_n$ is some gaussian noise $W_n \sim \mathcal{N}(0,1/n^2)$. here we don't care if $W_n,W_m$ are indepenpent, if you want you can consider the case $X_n = X+ \frac{W_1}{n}$, and clearly the noise $X_n - X$ tends to $0$ when $n \to \infty$ : $X_n$ converges in probability to $X$ $\endgroup$ – reuns Apr 2 '16 at 23:22
  • $\begingroup$ In this case the probability distribution of the difference is easy to deduce. I added the formulation of the CLT from proofwiki. In that case I can not how to deduce the probability distribution of the difference. $\endgroup$ – vkubicki Apr 3 '16 at 10:15
  • $\begingroup$ To Did: "there are many other situations where one can show convergence in probability without knowing this distribution". As I understand it, if the joint probability is not provided, there can be multiple distributions for $X_n - X$. So, you might not know it, but how can you know it is unique ? Maybe I do not understand something very fundamental :). $\endgroup$ – vkubicki Apr 3 '16 at 10:26
  • $\begingroup$ The example you added just muddies things beyond repair: note that the convergence in the CLT is a convergence in distribution which cannot be enhanced to a convergence in probability, even less to an almost sure convergence. Thus, the fact that you ask "how to deduce the expression of the probability distribution of $\frac {S_n - n \mu} {\sqrt {n \sigma^2} } - N \left({0, 1}\right)$" seems to indicate some deep misconception about CLT. (Unrelated: Please use @user to signal a comment to user user.) $\endgroup$ – Did Apr 3 '16 at 12:27

Regardless of the "technical limitations" to compute it,the definition as it stands is correct; intuitively it means that the probability of finding the a difference between $X_n$ and $X$ bigger than $\epsilon$ goes to $0$.

Now, in most cases to show convergence in probability you don't need the distribution of $X_n - X$; maybe you know that $X_n$ tends to $X$ almost surely (and this implies convergence in probability). Or you can bound the probability $P(|X_n - X| \ge \epsilon) \le a_n \to 0$, which would suffice too.

  • 1
    $\begingroup$ Thank you, your answer and Did comments helped a lot. $\endgroup$ – vkubicki Apr 3 '16 at 15:00


  • $X$ is uniformly distributed in $(0,1)$;
  • $X_1,X_2,X_3,\ldots$ are conditionally independent given $X$;
  • The conditional distribution of $X_n$ given $X$, for $n=1,2,3,\ldots$, is $\operatorname{Bernoulli}(X)$, i.e. $$\begin{cases} \Pr(X_n=1\mid X) = X, \\[3pt] \Pr(X_n = 0\mid X) = 1-X. \end{cases}$$

Let $\overline X_n = \dfrac{X_1+\cdots+X_n} n$. Then $\overline X_n$ converges almost surely to $X$, and therefore converges in probability to $X$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.