$\newcommand{\set}[1]{\{#1\}}$ $\newcommand{\mc}{\mathcal}$ $\newcommand{\P}{\mathbb P}$ $\newcommand{\N}{\mathbb N}$ $\newcommand{\E}{\mathbb E}$ $\newcommand{\mr}{\mathscr}$

I am looking for the proof of the theorem in Markov chain theory which roughly states that a recurrent Markov chain admit an essentially unique invariant measure (See the theorem at the end for the precise statement).

The theorem is stated here (Theroem 10) but no proof or a reference is provided.


Transition Probability Kernel

Let $S$ be a set and $\mathcal S$ be a $\sigma$-algebra on $S$. A transition probability kernel on $S$ is a map $K:S\times \mathcal S\to [0, 1]$ such that
$\bullet$ $K(x, \cdot)$ is a probability measure on $(S, \mathcal S)$ for each $x\in S$.
$\bullet$ The map $K(\cdot, A)$ is a measurable function from $S$ to $[0, 1]$ for each $A\in \mathcal S$.

We will refer to a measurable space $(S, \mc S)$ equipped with a transition probability kernel as a Markov chain.

One can think of $K(x, A)$ as the probability of jumping from $x$ to $A$ in one step. This viewpoint naturally given rise, for each $n\geq 1$, to a transition probability kernel $K^n:S\times \mathcal S\to [0, 1]$ as follows:

Let $\mr P(S)$ denote the set of all the probability measures in $S$. Define a map $K_\sharp:\mr P(S)\to \mr P(S)$ as $(K_\sharp\mu)(A)=\int_S K_x(A)\ d\mu(x)$ for all $\mu\in \mr P(S)$ and all $A\in \mc S$. Define $K^n$ as $K\circ (K_\sharp)^{n-1}$. We can thus think of $K^n(x, A)$ as the probability of jumping from $x$ to $A$ in $n$ steps. We refer to $K^n$ as the $n$-step transition kernel (arising out of $K$).

Forward Trajectories

Let $K$ be a transition probability kernel on $S$. For each $n\geq 0$, let $\Omega_n=\prod_{i=0}^n S$ and equip it with the product $\sigma$-algebra. Write $\P_x^0$ to denote the Dirac measure $\delta_x$ on $\Omega_0=S$.

We define a measure $\P_x^1$ on $\Omega_1$ as follows: $$ \P_x^1(E_0\times E_1) = \int_{E_0} P_y(E_1) \ d\P_x^0(y) $$ for all $E_0, E_1\in \mc S$. Intuitively, $\P_x^1(E_0\times E_1)$ is the probability that the Markov chain ``follows the trajectory $E_0\times E_1$" provided it starts at $x$. It is easy to verify that $\P_x^1$ is indeed a finitely additive measure on the algebra of measurable rectangles in $\Omega_1$ and thus by the Caratheodory extension theorem extends uniquely to a measure on the Borel $\sigma$-algebra of $\Omega_1$.

More generally, once $\P_x^n$ is constructed, we can construct $\P_x^{n+1}$ by declaring $$ \P_x^{n+1}(E_0\times \cdots \times E_{n+1}) = \int_{E_0\times \cdots \times E_n} P_{\omega_n}(E_{n+1})\ d\P_x^n(\omega) $$ for all $E_0 , \ldots, E_{n+1}\in \mc S$. Again, intuitively, $\P_x^{n+1}(E_0\times \cdots \times E_{n+1})$ is the probability that the Markov chain follows the trajectory $E_0\times \cdots \times E_{n+1}$ provided it starts at $x$. Note that the push-forward of $\P_x^{n}$ to $\Omega_{n-1}$ by the projection map $\Omega_n\to \Omega_{n-1}$ which kills the last coordinate is same as $\P_x^{n-1}$. So we have a unique probability measure $\P_x$ on the product space $\Omega=\prod_{n=0}^\infty S$ such that the push-forward of $\P_x$ to $\Omega_n$ under the projection map $\Omega\to \Omega_n$ which kills all coordinates beyond $n$ is $\P_x^n$. See Figure, where all the arrows are projection maps. The set $\Omega$ equipped with the product $\sigma$-algebra and the measure $\P_x$ can be thought of as the limit of the system $ \cdots \to \Omega_2\to\Omega_1\to\Omega_0$.

enter image description here

Irreducibility and Recurrence

Given a non-zero measure $\phi$ on $S$, we say that $K$ is $\phi$-irreducible if for all $A\in \mathcal S$ with $\phi(A)>0$ and all $x\in S$ we have that $K^n(x, A)>0$ for some $n\geq 1$.

For $A\in \mathcal S$, define a map $\eta_A:\Omega\to \N_0\cup{\infty}$ as $\eta_A(\omega)=|\set{n\in \N_0:\ \omega_n\in A}|$, where $\omega_n$ is the $n$-coordinate of $\omega$. We say that a $\phi$-irreducible chain is recurrent if $\E_{\P_x}[\eta_A]=\infty$ for all $x\in S$ whenever $\phi(A)>0$.

Invariant Measure

Given a transition probability kernel $K$ on a state space $S$, we say that a measure $\mu$ on $\mc S$ is invariant if $\mu(A)=\int_S K_x(A)\ d\mu(x)$ for all $A\in \mc S$.

The Theorem

Theorem 10 here states the following.

Theorem 1. A recurrent Markov chain has a unique invariant measure (up to a multiplicative constant).

However, no proof or a reference is provided.

  • $\begingroup$ Books on the theory of Markov chains usually present the proof. For example, Markov Chains - Gibbs Fields, Monte Carlo Simulation, and Queues by Pierre Brémaud. $\endgroup$ – Did Dec 20 '18 at 8:38
  • $\begingroup$ I saw the book but it considers state space as $\mathbb N$ but the question above probably asking on state-space as metric space? If I have mistaken, could you please cite the chapter and page number? $\endgroup$ – Marso Aug 14 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.