1
$\begingroup$

I understand that the Markov Chain generated by the Metropolis-Hastings algorithm satisfies the detailed balance condition, thus implying that the chain has a stationary distribution, $\pi(.)$, say.

However, I'm not sure how we can then apply the ergodic theorem to this Markov Chain? The ergodic theorem states that the chain must be positive recurrent, aperiodic and irreducible; but all we know about the Markov Chain generated by the MH algorithm is that it has $\pi(.)$ as its stationary distribution?

Thank you for your help!

$\endgroup$
0
$\begingroup$

Briefly, the criterion for the ergodicity of Markov Chain taking discrete values (finite or countable state space) does not extend to the continuous state spaces typically encountered in using the M-H algorithm.

You can easily see that the M-H algorithm specifies a Markov Chain because each value depends on the probability structure and the previous step. So the 'one-step dependence' of a Markov Chain is clear. However, in practice it is not so easy to know whether or how fast M-H chains converge.

In practice, one often uses graphical diagnostics to make sure a M-H chain is behaving as desired: 'History' plots of chain values (in each dimension separately) against chain steps (time) can show whether the process is 'mixing well', that is, freely moving about the state space instead of 'getting stuck' at particular values or regions. A plot of the 'autocorrelation function' (ACF) can reveal whether values widely-separated in time are nearly independent, as they should be in a useful M-H chain.

There is a trade-off between the sizes of the jumps the M-H chain makes when proposing new 'candidate' values, one the one hand, and the speed of convergence, on the other. If the jumps are too big, many candidates will be rejected. If the jumps are too small, the chain may not move freely througout the state space. By 'tuning' the algorithm, one hopes for a suitable compromise. In the process one gets pretty solid evidence whether the M-H chain is ergodic. Or, perhaps more crucially, whether the rate of convergence is sufficiently fast for the M-H algorithm to yield useful results.

This discussion of some practical issues in M-H convergence above is not intended to ignore the important theoretical issues. Most M-H chains used in practice are 'geometrically ergodic' and there are theoretical conditions for that. You can google 'geometric ergodicity of Metropolis-Hastings' to access the considerable amount of scholarly literature on this topic--much of it from the 1990s.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.