# Proving a Markov chain has a stationary measure

I'm in a measure-theoretic probability class and I am studying for an upcoming exam. Here is a problem from a book that I am studying from for this exam. But I am really not sure how to solve this problem since I am new to weak measures and representations.

Consider a Markov transition function $$P$$ on a compact space $$X$$. Prove that the corresponding Markov chain has at least one stationary measure.

I am given the following hint:

(Hint: Take an arbitrary initial measure $$\mu$$ and define $$\mu_n = (P^{*})^{n}\mu, n \geq 0$$. Prove that the sequence of measures defined by $$\eta_n = (\mu_0 + \cdots + \mu_{n-1})/n$$ is weakly compact and the limit of a subsequence is a stationary measure.)

I am familiar with many theorems, like Prokhorov Theorem, Riesz Representation Theorem, and more. But I'm really not sure how to solve this problem. I will really appreciate any help.

I found these notes (whose source is completely unrelated to the source of the problem) online, and they have been helpful to me: https://www.math.wisc.edu/~roch/grad-prob/gradprob-notes22.pdf

But I still cannot solve the problem.

• So what does Prokhorov's theorem tell you about the set of probability measures on $X$? – kimchi lover Dec 3 '19 at 4:55
• So is the collection of prob. measures on $X$ tight? – kimchi lover Dec 3 '19 at 5:06
• Take as $K_\epsilon$ your $X$: it is compact, and you know its probability. – kimchi lover Dec 3 '19 at 5:18
• I did not realize that. Okay, now I agree that the collection of probability measures on $X$ is tight, so by Prokhorov's Theorem, it's weakly compact – hom Dec 3 '19 at 5:40
• I'm still not able to make much progress on this. Could you please help me? @kimchilover – hom Dec 3 '19 at 6:28

Let $$\mathcal P(X)$$ denote the space whose elements are Borel probability measures on $$X$$, with the topology of weak convergence of measures. A special case of Prokhorov's Theorem implies that when $$X$$ is compact, this space $$\mathcal P(X)$$ is also compact. In particular, all infinite sequences in $$\mathcal P(X)$$ possess a limit point.
You have an infinite sequence in $$\mathcal P(X)$$, thus existence of a limit point means that there is a probability measure $$\mu\in\mathcal P(X)$$ such that some subsequence of the sequence you have written converges weakly to $$\mu$$.
Now fix any open set $$A\subseteq X$$ and consider $$\mu(A)-P^*\mu(A)$$. Let $$\{n_k\}$$ denote the convergent subsequence. Observe how $$|\mu(A)-P^*\mu(A)|=\lim_{k\to\infty}\frac{|\mu_0(A)-\mu_{n_k}(A)|}{n_k}=0,$$ since the difference of measures telescopes and $$|\mu_0(A)-\mu_{n_k}(A)|\leq 1$$.