I have a general question about Markov chain Monte Carlo. It can be illustrated by an example: Gibbs sampling. In Gibbs sampling, we prove that the Markov chain $M_t$ is stationary, where $M_t$ selects a random index and updates it. But in practice we usually select the index deterministically. From the Gibbs sampling wikipedia page:

In practice, the suffix $j$ is not chosen at random, and the chain cycles through the suffixes in order. In general this gives a non-stationary Markov process, but each individual step will still be reversible, and the overall process will still have the desired stationary distribution (as long as the chain can access all states under the fixed ordering).

Why is this true?

The more general question is that it seems that often the Markov chain which has the nice theoretical results is not precisely the one we use. How do we then justify that it works well enough for our purposes? (You could say I'm looking for a more general explanation of the quoted section above, that applies to more than just Gibbs sampling.)


1 Answer 1


Once you have reversibility of transition function $P$ w.r.t. to distribution $\pi$, that is $$ \forall_{A,B} \int_A P(x, B)d\pi(x) = \int_B P(y, A)d\pi(y), $$ you get that $\pi$ is a stationary distribution $$ \pi P(A) =\int_X P(x, A)d\pi(x) \overset{reversibility}= \int_A P(y, X)d\pi(y) = \int_A 1\ d\pi(y) = \pi(A). $$

  • $\begingroup$ Do you know a good reference for this material? Thanks $\endgroup$
    – Eric Auld
    Commented Oct 2, 2019 at 7:57
  • $\begingroup$ Unfortunately no. $\endgroup$
    – mbartczak
    Commented Oct 2, 2019 at 22:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .