Take the simple two state Markov process characterized by transitions $$ \begin{aligned} 0\rightarrow1 & \quad \text{ at rate } \quad \alpha\lambda \\ 1\rightarrow0 & \quad \text{ at rate } \quad \alpha. \end{aligned} $$

If we let $\alpha$ go to infinity, then we effectively have a stationary process. No matter what the initial distribution, the process at any future time has distribution $\pi.$ Is that correct? I know that $\pi=(\pi_0,\pi_1)=\left(\frac{1}{1+\lambda},\frac{\lambda}{1+\lambda}\right)$ irrespective of $\alpha.$

What if we introduce another state $2$ with transitions $$ \begin{aligned} 0\rightarrow2 & \quad \text{ at rate } \quad \gamma_{02} \\ 1\rightarrow2 & \quad \text{ at rate } \quad \gamma_{12} \\ 2\rightarrow1 & \quad \text{ at rate } \quad \beta \\ 2\rightarrow0 & \quad \text{ at rate } \quad \delta. \end{aligned} $$

Now call this $3$ state process $X_\alpha.$

Now what happens as we let $\alpha\rightarrow\infty?$ Presumably $X_\alpha\overset{\alpha\rightarrow\infty}{\longrightarrow} X$ where $X$ is the process which can be characterized with what follows.

We can think of $X$ as having only two states: $\{0,1\}$ and $2.$

The transitions of $X$ are $$\{0,1\}\rightarrow2 \quad \text{ at rate } \quad \gamma_{02} \pi_0+\gamma_{12} \pi_1$$ where $\pi_i$ is as described above, and $$2\rightarrow\{0,1\} \quad \text{ at rate } \quad \delta+\beta.$$

Question: Is it ok to think of $X$ in this way (as a two-state process)? When $X$ is not in state $2$, we know its distribution is $\pi$, yes?

What would be the best way to formalize the idea of $X_\alpha\overset{\alpha\rightarrow\infty}{\longrightarrow} X?$ I.e. what mode of convergence should be used?

  • $\begingroup$ This phenomenon of "aggregating" states is a major part of "Freidlin-Wentzell theory". In this first step, grouping $0$ and $1$ together, you have identified the first order cycles in the hierarchy of Freidlin cycles. You can find a great deal of detail on this subject (although mostly focused on the context of continuous space) in the book Random perturbations of dynamical systems. $\endgroup$ – Ian Jun 13 '16 at 18:35
  • $\begingroup$ By the way, you should keep in mind that usually we think about this situation in a sense inverted from what you have described here. Usually some or all of the transition rates of the process are going to zero. Then a different level of the Freidlin cycle hierarchy is appropriate depending on what time scale you are looking at. $\endgroup$ – Ian Jun 13 '16 at 18:54
  • $\begingroup$ One major example that is motivated by situations in physics is where you have $n$ states with "minimum energies" $V_i \geq 0$ and "saddle energies" $V_{ij}=V_{ji} \geq 0$. Here the saddle energies can be $+\infty$ but the minimum energies cannot. The transition rate from $i$ to $j$ here is proportional to or at least logarithmically equivalent to $e^{-\beta(V_{ij}-V_i)}$ where $\beta>0$ is a large parameter. $\endgroup$ – Ian Jun 13 '16 at 19:13
  • $\begingroup$ Now generically, for any state $i$, there is a state most asymptotically accessible from $i$. Following such paths, you will eventually find a loop (though it may not come back to $i$). These loops, and any isolated states not in any loop, are the first order Freidlin cycles. You obtain higher order cycles by defining a new process on the macrostates (as you did) and repeating the procedure. $\endgroup$ – Ian Jun 13 '16 at 19:13
  • 1
    $\begingroup$ $P(X(t)=0)$ makes no sense, because $X$ does not distinguish between $0$ and $1$. $P(X_\alpha(t)=0) \to \pi_0$ (where $\pi_0$ is defined as in your question) is also not necessarily true because that neglects the possibility that it is $2$. $P(X_\alpha(t)=0 \mid X_\alpha(t) \in \{ 0,1 \}) \to \pi_0$ is correct. $\endgroup$ – Ian Jun 15 '16 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.