Okay so I recently learned about three types of convergence in probability. Convergence in distribution, convergence in probability and almost sure convergence.

I have the definitions memorised but I am struggling to understand what they mean in essence, why we need all these different types of convergence and a few technical details. Here are my concerns.

1) In two of these convergences we require that a certain limit is satisfied. Take for example convergence in probability it says

$X_1,X_2,...\implies X$ in probability if for all $\epsilon>0:\lim_{n \rightarrow \infty} P(|X_n-X|>\epsilon)=0$.

Now in my mind intuitively this is saying that as we go further and further along the sequence of random variables ($n \rightarrow \infty$) the probability that $X_n$ differs from $X$ by any positive amount becomes arbitrarily small.

But I don't understand what it means for $|X_n-X|$ to become small since these are random variables which are themselves functions. How exactly do we measure distance between these two functions? I have seen some metrics of spaces of functions but I don't think that is what is being used here.

Would it just mean that $|X_n(w)-X(w)|$ gets small for any $w$ in the sample space? In which case why don't we specify the definition as

$X_1,X_2,...\implies X$ in probability if for all $\epsilon>0$ and for all $w$ in the sample space :$\lim_{n \rightarrow \infty} P(|X_n(w)-X(w)|>\epsilon)=0$?

2) Almost sure convergence says $X_1,... \rightarrow X$ if we have $P(\lim_{n \rightarrow \infty} X_n=X)=1$ again what does this mean? Is it saying that $X_n(w)=X(w)$ in the limit as $n$ tends to infinity for all $w$ in the sample space.

3) Why do we need both almost sure and convergence in probability? I know that almost sure is stronger and thus I suppose I have answered my own question but I don't really see why. To me almost sure says that $X_n$ becomes the same as $X$ as we let $n$ become very large and convergence in probability to me seems to say $X_n$ can't be different from $X$ as $n$ becomes large this seems like two ways to say the same thing.

I think that is all I have at the moment hopefully I have expressed my concerns in a way that makes it somewhat easy to see what I don't get.

If anyone could help clarify that would be great.


By the way I have met these definitions in my lectures but nobody in our year has covered measure theory yet so I wonder if I am not truly meant to understand these definitions yet.

  • 2
    $\begingroup$ Realize that $|X(\omega)-X(\omega)|$ is just a real number, so that $P(|X(\omega)-X(\omega)|>\epsilon)$ makes no sense. $\endgroup$ – drhab Apr 10 '17 at 9:58


It does not directly mean that $|X-X_n|$ gets small. It is the set $\{|X-X_n|>\epsilon\}:=\{\omega\in\Omega\mid |X(\omega)-X_n(\omega)|>\epsilon\}$ that gets small. This for every $\epsilon>0$.

These sets are measured by means of the probability measure.


You are almost correct in your thinking. If $X_n(\omega)$ converges to $X(\omega)$ for every $\omega\in\Omega$ then indeed $X_n\to X$ almost surely. However, for this it is only demanded that the set $\{\omega\mid X_n(\omega)\text{ converges to }X(\omega)\}$ has measure $1$. In words: it must be the case for "almost every" $\omega\in\Omega$.


Good and recognizable question! To get hold on the difference you should take a look at examples where $X_n\to X$ in probability, but not almost surely. WLOG you can focus on the special case where $X=0$.

This example might help. Take $\Omega=[0,1]$ equipped with Lebesgue measure on Borel sets. For $n=1,2,\dots$ and $k=1,\dots,n$ define $Y_{n,k}$ as the characteristic function of set $[\frac{k-1}{n},\frac{k}{n}]$. Now look at the sequence $Y_{1,1},Y_{2,1},Y_{2,2},Y_{3,1},Y_{3,2},Y_{3,3},Y_{4,1},\dots$. Observe that for every $\omega\in[0,1]$ the sequence does not converge. However, we do have $\lambda\{|Y_{n,k}|\geq\epsilon\}\leq\frac1{n}$ for every $\epsilon>0$, so that there is convergence in probability.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.