Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

As I find some definition says: Ergodic = irreducible. And then Irreducible + aperiodic + positive gives Regular Markov chain.

A Markov chain is called an ergodic chain if it is possible to go from every state to every state (not necessarily in one move).

Ergodic Markov chains are also called irreducible.

A Markov chain is called a regular chain if some power of the transition matrix has only positive elements.

Reference http://www.math.dartmouth.edu/archive/m20x06/public_html/Lecture15.pdf

However, in others the Regular Markov chain concept doesn't exist. And Ergodic replaces Regular in all literate.

So if our Markov chain is aperiodic, irreducible, and positive recurrent (all the ones we use in Bayesian statistics usually are), then it is ergodic.

we shall sometimes refer to an irreducible, aperiodic Markov chain as ergodic

Reference http://www.people.fas.harvard.edu/~plam/teaching/methods/mcmc/mcmc_print.pdf

http://www.cs.berkeley.edu/~sinclair/cs294/n2.pdf

share|improve this question
    
Can you maybe give references for these different definitions? Then maybe someone will be able to make sense of them and explain away the confusion. –  M Turgeon Jun 1 '12 at 15:14

2 Answers 2

up vote 2 down vote accepted

I prefer the first definition by far. I relate the question to ergodic theory, as seems appropriate, and assume that the chain hass finitely many possible values, so as to not bother with positive recurrence.

Let us give a finite state space $A$, Then all the possible sequences of element in $A$ by $X:=A^{\mathbb{N}}$. let us define a transformation $\sigma$ on $X$ by $(\sigma x)_n = x_{n+1}$ on $X$. For $x$ in $A$, we have $x_n = (\sigma^n x)_0$. In other words, by applying the transformation, I can read the successive values of a given sequence. Now, if we give ourselves a probability measure $\mu$ on $A$, the random sequence $(x_n)_{n \geq 0} = ((\sigma^n x)_0)_{n \geq 0}$ is a stochastic process with values in $A$. If $\mu$ is well-chosen (i.e. satisfies the usual conditions on cylinders), then this stochastic process is a Markov chain.

The construction may look a bit confusing. However, if you forget about $\sigma$, it is what is done more or less informally when one defines Markov chains (that is the construction may be hidden, but it is there).

Now, we can consider a Markov chain as a dynamical system $(X, \sigma)$ together with a probability measure $\mu$. We can use the definitions of ergodic theory, and what we get in the end is that:

  • the system is ergodic (in the sense of ergodic theory) if and only if the Markov chain is irreducible;
  • the system is mixing if and only if the Markov chain is irreducible and aperiodic.

So these are two very different conditions, and aperiodicity does not correspond to ergodicity. As a corollary, one can apply ergodic theorems to Markov chains with no need for aperiodicity.

share|improve this answer
    
How is Markov chain splitting technique useful for inferring ergodicity of a Markov Chain?Assume that I am working with general state space (uncountable say $R^{N}$ but time is discrete. I want to show that the Markov Process is ergodic. I guess that it suffices to show that it is Harris recurrent. To show Harris recurrence I guess that it suffices to show there exists an atom (obtained via splitting the chain after using minorization criteria) the return time (or hitting time) to which has finite mean. –  user24367 Jun 18 '13 at 17:37

I consider also just Markov chains with finite state space, as I do not know much about ergodic theory with infinite measures.

Irreducibility of the transition graph of the state space means that a sample path cannot get trapped in smaller subsets of the state space, since one can go from ererywhere to everywhere. This implies, but one has to work this out, that the whole chain is ergodic: (almost) every path of the chain shows statistical stable behavior, that roughly means that the frequencies visiting any state of the space converge. Dynamically speaking, there exists an invariant probability measure on the sapce of all paths. The opposite is obviously true: if there is an invariant measure (that is supported by the whole state space) then one can go from every state to every state.

Aperiodic transition graphs imply roughly speaking that there are no periodic phenomena hidden in the dynamic: the recurrence times of any point have greatest common divisor 1; more generally the n- step-transition matrix has positive entries for suitable large times n. This mateix property is usually used to prove convergence of the probabilities at time n to a single distribution called equilibrium as n tends to infinity (read the Perron Frobenius theorem). The chain is then called mixing. Mixing is stronge than ergodic: it implies that the future states become even asymptotically independent of the initial state.

I personally never heard of regular transition graphs - but the matrix property referred in your link is again the same property one uses to show converge to equilibrium. I guess, but I don't know, that it is just a formal rather useless definition, since the only regular but not mixing chains I can think of can be decomposed in a suitable sense into mixing chains.

Hope this helps.

share|improve this answer
1  
This lacks some rigor. Most notably, having an invariant probability measure supported by the whole space does not imply (and by far) that the system is transitive. You need the measure, well, to be ergodic. –  D. Thomine Jun 5 '12 at 7:26
    
How is Markov chain splitting technique useful for inferring ergodicity of a Markov Chain?Assume that I am working with general state space (uncountable say $R^{N}$ but time is discrete. I want to show that the Markov Process is ergodic. I guess that it suffices to show that it is Harris recurrent. To show Harris recurrence I guess that it suffices to show there exists an atom (obtained via splitting the chain after using minorization criteria) the return time (or hitting time) to which has finite mean. –  user24367 Jun 18 '13 at 17:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.