I often get confused about
- when a Markov chain has an equilibrium distribution;
- when this equilibrium distribution is unique;
- which starting states converge to the equilibrium distribution; and
- how finite and countably infinite Markov chains different with respect to the above.
(Google isn't quite clearing up my confusion.) Is the following correct/am I missing anything?
An irreducible Markov chain (finite or countably infinite) has a unique equilibrium distribution if and only if all states are positive recurrent. (What about reducible Markov chains? A reducible Markov chain has a non-unique equilibrium distribution iff all states are positive recurrent?) However, not all starting states necessarily converge to the unique equilibrium, unless the Markov chain is also aperiodic; that is, an irreducible Markov chain converges to its unique equilibrium regardless of initial state, if and only if all states are positive recurrent and aperiodic.