How is the Markov Chain generated by the Metropolis-Hastings Algorithm Ergodic? I understand that the Markov Chain generated by the Metropolis-Hastings algorithm satisfies the detailed balance condition, thus implying that the chain has a stationary distribution, $\pi(.)$, say. 
However, I'm not sure how we can then apply the ergodic theorem to this Markov Chain? The ergodic theorem states that the chain must be positive recurrent, aperiodic and irreducible; but all we know about the Markov Chain generated by the MH algorithm is that it has $\pi(.)$ as its stationary distribution?
Thank you for your help!
 A: Briefly, the criterion for the ergodicity of Markov Chain taking discrete
values (finite or countable state space) does not extend to the continuous
state spaces typically encountered in using the M-H algorithm.
You can easily see that the M-H algorithm specifies a Markov Chain because
each value depends on the probability structure and the previous step.
So the 'one-step dependence' of a Markov Chain is clear. However, in practice it is
not so easy to know whether or how fast M-H chains converge. 
In practice, one often uses graphical diagnostics to make sure a M-H chain
is behaving as desired: 'History' plots of chain values (in each dimension
separately) against chain steps (time) can show whether the process is
'mixing well', that is, freely moving about the state space instead
of 'getting stuck' at particular values or regions. A plot of the 'autocorrelation
function' (ACF) can reveal whether values widely-separated in time
are nearly independent, as they should be in a useful M-H chain.
There is a trade-off between the sizes of the jumps the M-H chain makes
when proposing new 'candidate' values, one the one hand, and the speed
of convergence, on the other. If the jumps are too big, many candidates
will be rejected. If the jumps are too small, the chain may not move
freely througout the state space. By 'tuning' the algorithm, one hopes
for a suitable compromise. In the process one gets pretty solid evidence
whether the M-H chain is ergodic. Or, perhaps more crucially, whether
the rate of convergence is sufficiently fast for the M-H algorithm
to yield useful results. 
This discussion of some practical issues in M-H convergence above is not intended to ignore the important theoretical issues. Most M-H chains used in practice are
'geometrically ergodic' and there are theoretical conditions for that.
You can google 'geometric ergodicity of Metropolis-Hastings' to access
the considerable amount of scholarly literature on this topic--much of it
from the 1990s.
