I'm just getting started with Markov chains and have a few simple questions:

  1. Is it possible to define a period for a reducible Markov chain? If so, how?
  2. Can we define balance equations and a stationary distribution for reducible Markov chains?



For (1), note that each state has its own period. One can show that any two communicating states have the same period, so for an irreducible Markov chain, all states have the same period, and thus one may call this the period of the chain. For a reducible chain, non-communicating states may have different periods, so the "period of the chain" is not well defined.

For (2), the notion of stationary distribution still makes sense for a non-reducible Markov chain, but in general there may be more than one of them. Each of them may or may not satisfy the balance equations.

The theory of reducible Markov chains effectively reduces to that for irreducible chains, because of the following fact:

Proposition. The state space $S$ of any Markov chain may be partitioned as $S = T \cup \bigcup_i C_i$, where the $C_i$ are closed and irreducible, and for every $x \in T$ there exists $y \in S$ such that $x \to y$ but $y \not\to x$.

So each $C_i$ can be considered an irreducible Markov chain in its own right. Each $C_i$ has a well-defined period $p_i$, but of course these periods may be different for different $i$. Each $C_i$ has at most one stationary distribution; if $C_i$ has stationary distribution $\pi_i$, it extends to a stationary distribution for $S$ by setting $\pi_i(x)=0$ for $x \notin C_i$. Thus the chain has (potentially) one stationary distribution for each $C_i$, and one can get more of them by taking convex combinations $\pi = \sum_i a_i \pi_i$ where $a_i \ge 0$, $\sum_i a_i = 1$.

The states of $T$ are all transient, so they don't really enter into the limiting behavior of the chain. For instance, any stationary distribution $\pi$ for $S$ must have $\pi(x) = 0$ for $x \in T$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy