A counterexample of reducible Markov chains about Birkhoff's ergodic theorem

In R.Durrett's boos Probability Theory and Examples(5th). There is an example 6.2.4(CH6.2 P301),

"Let $X_{n}$ be an $\textbf{irreducible}$ Markov chain on a countable state space that has a stationary distribution $\pi$..., and applying the ergodic theorem, $$\frac{1}{n} \sum_{m=0}^{n-1} f(X_{m}) \longrightarrow \sum_{x}f(x)\pi(x)."$$

I want to know if there exists a counterexample removing the condition "irreducible", in other words, how about the reducible Markov chain? In this counterexample, the Birkhoff's ergodic theorem does not hold.

• "removing the condition "irreducible", in other words, how about the reducible and periodic Markov chain" You mean: reducible OR periodic.
– Did
Apr 17, 2018 at 17:56
• yes, reducible or periodic...
– user469065
Apr 18, 2018 at 3:19
• @Did But I just want to construct a reducible example.
– user469065
Apr 18, 2018 at 3:33

1 Answer

Take the example of a Markov chain with three states $0$, $1$ and $2$, with transition probabilities

This Markov chain has precisely two irreducible components corresponding to the two absorbing states $1$ and $2$. Every distribution $\pi$ with $\pi(1)+\pi(2)=1$ is stationary. But depending on the initial state, the Markov chain will eventually be absorbed to one of the two absorbing states. Therefore, the ergodic averages $(1/n)\sum_{m=0}^{n-1}f(X_m)$ will converge almost surely to a random variable with two possible values $f(1)$ and $f(2)$, and in particular will not converge to a constant unless $f(1)=f(2)$.

In general, if a Markov chain is not irreducible, then it may have $0$, $1$ or more irreducible components, and each of these components may or may not support a stationary distribution. The convergence of the ergodic averages will then depend on whether or not the chain ends up in an irreducible component and whether or not that irreducible component supports a stationary distribution ($\equiv$ is positively recurrent). If $C_i$ is an irreducible component with stationary distribution $\pi_i$, then conditioned on the event that the Markov chain ends up in $C_i$, the ergodic averages will converge almost surely to the mean $\sum_x\pi_i(x)f(x)$.

Just a final remark that for the convergence of the ergodic averages, aperiodicity is irrelevant.

• Thanks for your example.
– user469065
Apr 18, 2018 at 3:33