First of all I apologize if some of the below is not clear, as I'm trying to make this as precise as possible but am not a professional mathematician...corrections or questions in the comments are much appreciated and I will correct per any guidance I receive.

Let me try to precisely define what I mean by a Markov Chain approximation to a real-valued univariate stochastic process :

A real-valued univariate stochastic process $X_t(\omega): \Omega \to \mathbb{R}$ can be approximated as a (possibly time inhomogenous) finite-state Markov Chain $M$ if there exists a sequence of finite-state-space Markov Chain models $M_n$ such that $$\forall t>0 \lim\limits_{n \to \infty} P_{X_t}[\mathrm{Range(M_{n,t})}\;\triangle \;\mathrm{Range(X_{t}})] = 0\;\mathrm{and}$$ $$\forall t>0\lim\limits_{n \to \infty}|F(M_{n,t}\leq x) - F(X_t\leq x)|=0 \;\forall x \in \mathrm{Range(X_{t}}) $$

Basically, it seems that a great many stochastic processes that model real-world dynamics can be modeled as a limit of such a sequence of finite Markov Chains.

The reason I care about this is that in applications, we often don't need absolute precision, just adequate agreement. If I can model a stochastic process (e.g., using the language of SDEs) then convert the problem to an approximation using the (much simpler and more familiar) language of Markov Chains, then it seems that a much larger range of stochastic process become amenable to analysis and numerical solutions by non-specialists in stochastic processes and practitioners.

In particular, I could use my familiarity with Markov Chains to (a) derive approximate results and (b) possibly apply limiting arguments to derive the exact results without recourse to advanced mathematics like measure theory and real analysis.

Anyway, that is my motivation for asking this, and I'd like to know the limits of this approach/idea.

Main Question: What properties of $X_t$ are necessary or sufficient (or both) for the above approximation to hold (assuming my definition of approximation is mathematically consistent (no internal contradictions or missing assumptions) and sufficient for justifying Markov Chain approximations)

  • 1
    $\begingroup$ I'm not sure this is sufficiently well-defined to be answered. You might just want to generally look into continuum limits of Markov chains. One useful, quite quantitative result that I have used is a theorem due to Kurtz, which says that for "density-dependent" Markov chains, they are approximated by a certain SDE in a "large size" limit, with the error being significantly smaller than the corresponding deterministic limit (error on the order of $\ln(A)/A$ instead of $A^{-1/2}$, where $A$ is a large parameter). $\endgroup$ – Ian Sep 8 '16 at 13:45
  • 1
    $\begingroup$ This theorem and its proof, along with some other related results, can be found in "Strong approximation theorems for density-dependent Markov chains". Of course this is really going in the opposite direction of what you want to do. $\endgroup$ – Ian Sep 8 '16 at 13:47
  • 1
    $\begingroup$ For more of the forward direction of what you want to do, you can look at "stochastic network" approximations of metastable SDEs. Essentially the idea here is to look at an SDE model which takes very short times to move between neighborhoods of certain points (e.g. potential energy minima, in chemical applications) and moves between those points on paths which are close to particular paths (e.g. minimax energy paths in chemical applications). I've worked in this area a bit as well. $\endgroup$ – Ian Sep 8 '16 at 13:48
  • $\begingroup$ @Ian thanks for your input. I'll take a look at these sources to see if I can refine my problem/question. In the meantime, any suggestions for constraining my question further would be appreciated. $\endgroup$ – user237392 Sep 8 '16 at 13:49
  • 1
    $\begingroup$ For example, if you do this with 1D Brownian motion using a uniform grid, you get the familiar symmetric random walk approximation to Brownian motion. $\endgroup$ – Ian Sep 8 '16 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy