I have two books on stochastic processes.

In one book, it says that the limiting matrix is possible to find if the matrix is regular, that is, if for some $n$ $P^n$ has only positive values. The other book says that the limiting values are possible to find if the Markov chain is recurrent, irreducible and aperiodic; it is then called ergodic.

Does this then hold:

aperiodic + irreducible $\Leftrightarrow$ ergodic $\Leftrightarrow$ regular?

And is there any difference whether it is a finite-state chain or not?

  • $\begingroup$ Is late now I guess but it woud be great to have the name of the books $\endgroup$ – Javier Dec 1 '18 at 17:26

For a finite MC it holds that

aperiodic + irreducible $\Leftrightarrow$ ergodic $\Leftrightarrow$ regular

as you expected. For an infinite MC it holds that

aperiodic + irreducible + positive recurrent $\Leftrightarrow$ ergodic,

and being "regular" in the infinite setting would require a more precise definition.

................................ explanations following ................................

For every finite or inifinite Markov chain (MC) it holds that

$aperiodic + irreducible + positive~recurrent \Leftrightarrow ergodic$.

See for example here for a proof. For every finite MC, irreducibility already implies positive recurrence, see here for a proof.

Further, for every finite MC we have that

$aperiodic + irreducible \Leftrightarrow regular$.

Proof sketch: the definition of a finite irreducible MC gives that $\forall i, j \in \Omega : \exists k > 0 : P^k[i,j] > 0$. However, there might be no $k$ such that all entries are simultaneously positive - due to periodicities. But if the chain is additionally aperiodic, it follows that $\exists k > 0 : \forall i, j \in \Omega : P^k[i,j] > 0$, which matches your definition of being regular.

Finally, I don't see a canonical way how you would generalize the property "regular" to infinite Markov chains. So, I just ignore the term "regular" for infinite chains here.

  • $\begingroup$ Thank you very much!, it cleared the situation up for me! $\endgroup$ – user119615 Jan 21 '14 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.