# Recurrent Markov chain has an invariant measure

$$\newcommand{\set}{\{#1\}}$$ $$\newcommand{\mc}{\mathcal}$$ $$\newcommand{\P}{\mathbb P}$$ $$\newcommand{\N}{\mathbb N}$$ $$\newcommand{\E}{\mathbb E}$$ $$\newcommand{\mr}{\mathscr}$$

I am looking for the proof of the theorem in Markov chain theory which roughly states that a recurrent Markov chain admit an essentially unique invariant measure (See the theorem at the end for the precise statement).

The theorem is stated here (Theroem 10) but no proof or a reference is provided.

# Definitions

## Transition Probability Kernel

Let $$S$$ be a set and $$\mathcal S$$ be a $$\sigma$$-algebra on $$S$$. A transition probability kernel on $$S$$ is a map $$K:S\times \mathcal S\to [0, 1]$$ such that
$$\bullet$$ $$K(x, \cdot)$$ is a probability measure on $$(S, \mathcal S)$$ for each $$x\in S$$.
$$\bullet$$ The map $$K(\cdot, A)$$ is a measurable function from $$S$$ to $$[0, 1]$$ for each $$A\in \mathcal S$$.

We will refer to a measurable space $$(S, \mc S)$$ equipped with a transition probability kernel as a Markov chain.

One can think of $$K(x, A)$$ as the probability of jumping from $$x$$ to $$A$$ in one step. This viewpoint naturally given rise, for each $$n\geq 1$$, to a transition probability kernel $$K^n:S\times \mathcal S\to [0, 1]$$ as follows:

Let $$\mr P(S)$$ denote the set of all the probability measures in $$S$$. Define a map $$K_\sharp:\mr P(S)\to \mr P(S)$$ as $$(K_\sharp\mu)(A)=\int_S K_x(A)\ d\mu(x)$$ for all $$\mu\in \mr P(S)$$ and all $$A\in \mc S$$. Define $$K^n$$ as $$K\circ (K_\sharp)^{n-1}$$. We can thus think of $$K^n(x, A)$$ as the probability of jumping from $$x$$ to $$A$$ in $$n$$ steps. We refer to $$K^n$$ as the $$n$$-step transition kernel (arising out of $$K$$).

## Forward Trajectories

Let $$K$$ be a transition probability kernel on $$S$$. For each $$n\geq 0$$, let $$\Omega_n=\prod_{i=0}^n S$$ and equip it with the product $$\sigma$$-algebra. Write $$\P_x^0$$ to denote the Dirac measure $$\delta_x$$ on $$\Omega_0=S$$.

We define a measure $$\P_x^1$$ on $$\Omega_1$$ as follows: $$\P_x^1(E_0\times E_1) = \int_{E_0} P_y(E_1) \ d\P_x^0(y)$$ for all $$E_0, E_1\in \mc S$$. Intuitively, $$\P_x^1(E_0\times E_1)$$ is the probability that the Markov chain follows the trajectory $$E_0\times E_1$$" provided it starts at $$x$$. It is easy to verify that $$\P_x^1$$ is indeed a finitely additive measure on the algebra of measurable rectangles in $$\Omega_1$$ and thus by the Caratheodory extension theorem extends uniquely to a measure on the Borel $$\sigma$$-algebra of $$\Omega_1$$.

More generally, once $$\P_x^n$$ is constructed, we can construct $$\P_x^{n+1}$$ by declaring $$\P_x^{n+1}(E_0\times \cdots \times E_{n+1}) = \int_{E_0\times \cdots \times E_n} P_{\omega_n}(E_{n+1})\ d\P_x^n(\omega)$$ for all $$E_0 , \ldots, E_{n+1}\in \mc S$$. Again, intuitively, $$\P_x^{n+1}(E_0\times \cdots \times E_{n+1})$$ is the probability that the Markov chain follows the trajectory $$E_0\times \cdots \times E_{n+1}$$ provided it starts at $$x$$. Note that the push-forward of $$\P_x^{n}$$ to $$\Omega_{n-1}$$ by the projection map $$\Omega_n\to \Omega_{n-1}$$ which kills the last coordinate is same as $$\P_x^{n-1}$$. So we have a unique probability measure $$\P_x$$ on the product space $$\Omega=\prod_{n=0}^\infty S$$ such that the push-forward of $$\P_x$$ to $$\Omega_n$$ under the projection map $$\Omega\to \Omega_n$$ which kills all coordinates beyond $$n$$ is $$\P_x^n$$. See Figure, where all the arrows are projection maps. The set $$\Omega$$ equipped with the product $$\sigma$$-algebra and the measure $$\P_x$$ can be thought of as the limit of the system $$\cdots \to \Omega_2\to\Omega_1\to\Omega_0$$. ## Irreducibility and Recurrence

Given a non-zero measure $$\phi$$ on $$S$$, we say that $$K$$ is $$\phi$$-irreducible if for all $$A\in \mathcal S$$ with $$\phi(A)>0$$ and all $$x\in S$$ we have that $$K^n(x, A)>0$$ for some $$n\geq 1$$.

For $$A\in \mathcal S$$, define a map $$\eta_A:\Omega\to \N_0\cup{\infty}$$ as $$\eta_A(\omega)=|\set{n\in \N_0:\ \omega_n\in A}|$$, where $$\omega_n$$ is the $$n$$-coordinate of $$\omega$$. We say that a $$\phi$$-irreducible chain is recurrent if $$\E_{\P_x}[\eta_A]=\infty$$ for all $$x\in S$$ whenever $$\phi(A)>0$$.

## Invariant Measure

Given a transition probability kernel $$K$$ on a state space $$S$$, we say that a measure $$\mu$$ on $$\mc S$$ is invariant if $$\mu(A)=\int_S K_x(A)\ d\mu(x)$$ for all $$A\in \mc S$$.

# The Theorem

Theorem 10 here states the following.

Theorem 1. A recurrent Markov chain has a unique invariant measure (up to a multiplicative constant).

However, no proof or a reference is provided.

• Books on the theory of Markov chains usually present the proof. For example, Markov Chains - Gibbs Fields, Monte Carlo Simulation, and Queues by Pierre Brémaud. – Did Dec 20 '18 at 8:38
• I saw the book but it considers state space as $\mathbb N$ but the question above probably asking on state-space as metric space? If I have mistaken, could you please cite the chapter and page number? – Marso Aug 14 at 10:14