Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two books on stochastic processes.

In one book, it says that the limiting matrix is possible to find if the matrix is regular, that is, if for some $n$ $P^n$ has only positive values. The other book says that the limiting values are possible to find if the Markov chain is recurrent, irreducible and aperiodic; it is then called ergodic.

Does this then hold:

aperiodic + irreducible $\Leftrightarrow$ ergodic $\Leftrightarrow$ regular?

And is there any difference whether it is a finite-state chain or not?

share|cite|improve this question

For a finite MC it holds that

aperiodic + irreducible $\Leftrightarrow$ ergodic $\Leftrightarrow$ regular

as you expected. For an infinite MC it holds that

aperiodic + irreducible + positive recurrent $\Leftrightarrow$ ergodic,

and being "regular" in the infinite setting would require a more precise definition.

................................ explanations following ................................

For every finite or inifinite Markov chain (MC) it holds that

$aperiodic + irreducible + positive~recurrent \Leftrightarrow ergodic$.

See for example here for a proof. For every finite MC, irreducibility already implies positive recurrence, see here for a proof.

Further, for every finite MC we have that

$aperiodic + irreducible \Leftrightarrow regular$.

Proof sketch: the definition of a finite irreducible MC gives that $\forall i, j \in \Omega : \exists k > 0 : P^k[i,j] > 0$. However, there might be no $k$ such that all entries are simultaneously positive - due to periodicities. But if the chain is additionally aperiodic, it follows that $\exists k > 0 : \forall i, j \in \Omega : P^k[i,j] > 0$, which matches your definition of being regular.

Finally, I don't see a canonical way how you would generalize the property "regular" to infinite Markov chains. So, I just ignore the term "regular" for infinite chains here.

share|cite|improve this answer
Thank you very much!, it cleared the situation up for me! – user119615 Jan 21 '14 at 12:01

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.