Types of convergence in probability confusion?

Okay so I recently learned about three types of convergence in probability. Convergence in distribution, convergence in probability and almost sure convergence.

I have the definitions memorised but I am struggling to understand what they mean in essence, why we need all these different types of convergence and a few technical details. Here are my concerns.

1) In two of these convergences we require that a certain limit is satisfied. Take for example convergence in probability it says

$X_1,X_2,...\implies X$ in probability if for all $\epsilon>0:\lim_{n \rightarrow \infty} P(|X_n-X|>\epsilon)=0$.

Now in my mind intuitively this is saying that as we go further and further along the sequence of random variables ($n \rightarrow \infty$) the probability that $X_n$ differs from $X$ by any positive amount becomes arbitrarily small.

But I don't understand what it means for $|X_n-X|$ to become small since these are random variables which are themselves functions. How exactly do we measure distance between these two functions? I have seen some metrics of spaces of functions but I don't think that is what is being used here.

Would it just mean that $|X_n(w)-X(w)|$ gets small for any $w$ in the sample space? In which case why don't we specify the definition as

$X_1,X_2,...\implies X$ in probability if for all $\epsilon>0$ and for all $w$ in the sample space :$\lim_{n \rightarrow \infty} P(|X_n(w)-X(w)|>\epsilon)=0$?

2) Almost sure convergence says $X_1,... \rightarrow X$ if we have $P(\lim_{n \rightarrow \infty} X_n=X)=1$ again what does this mean? Is it saying that $X_n(w)=X(w)$ in the limit as $n$ tends to infinity for all $w$ in the sample space.

3) Why do we need both almost sure and convergence in probability? I know that almost sure is stronger and thus I suppose I have answered my own question but I don't really see why. To me almost sure says that $X_n$ becomes the same as $X$ as we let $n$ become very large and convergence in probability to me seems to say $X_n$ can't be different from $X$ as $n$ becomes large this seems like two ways to say the same thing.

I think that is all I have at the moment hopefully I have expressed my concerns in a way that makes it somewhat easy to see what I don't get.

If anyone could help clarify that would be great.

Thanks!

By the way I have met these definitions in my lectures but nobody in our year has covered measure theory yet so I wonder if I am not truly meant to understand these definitions yet.

• Realize that $|X(\omega)-X(\omega)|$ is just a real number, so that $P(|X(\omega)-X(\omega)|>\epsilon)$ makes no sense. – drhab Apr 10 '17 at 9:58

1)

It does not directly mean that $|X-X_n|$ gets small. It is the set $\{|X-X_n|>\epsilon\}:=\{\omega\in\Omega\mid |X(\omega)-X_n(\omega)|>\epsilon\}$ that gets small. This for every $\epsilon>0$.

These sets are measured by means of the probability measure.

2)

You are almost correct in your thinking. If $X_n(\omega)$ converges to $X(\omega)$ for every $\omega\in\Omega$ then indeed $X_n\to X$ almost surely. However, for this it is only demanded that the set $\{\omega\mid X_n(\omega)\text{ converges to }X(\omega)\}$ has measure $1$. In words: it must be the case for "almost every" $\omega\in\Omega$.

3)

Good and recognizable question! To get hold on the difference you should take a look at examples where $X_n\to X$ in probability, but not almost surely. WLOG you can focus on the special case where $X=0$.

This example might help. Take $\Omega=[0,1]$ equipped with Lebesgue measure on Borel sets. For $n=1,2,\dots$ and $k=1,\dots,n$ define $Y_{n,k}$ as the characteristic function of set $[\frac{k-1}{n},\frac{k}{n}]$. Now look at the sequence $Y_{1,1},Y_{2,1},Y_{2,2},Y_{3,1},Y_{3,2},Y_{3,3},Y_{4,1},\dots$. Observe that for every $\omega\in[0,1]$ the sequence does not converge. However, we do have $\lambda\{|Y_{n,k}|\geq\epsilon\}\leq\frac1{n}$ for every $\epsilon>0$, so that there is convergence in probability.