Looking around I have found lots of material on continuous time Markov processes on finite or countable state spaces, say $\{0,1,\ldots,J\}$ for some $J\in\mathbb{N}$ or just $\mathbb{N}$. Similarly I have earlier worked with (discrete time) Markov chains on general state spaces, following the modern classic by Meyn & Tweedie.

My question concerns monographs on continuous time Markov processes on general state spaces, say some subset of $\mathbb{R}^k$, $k\in\mathbb{N}$. Are there any good references - preferably but not necessarily suited for an ambitious master student - on this topic?

  • $\begingroup$ Then, it sounds like you need to write this monograph. $\endgroup$ – Michael Oct 8 '16 at 2:24
  • $\begingroup$ As a first cut, if you assume the transition rates $\nu(x)$ out of each state $x$ are bounded, so that there are values $\nu_{min}, \nu_{max}$ such that $0 < \nu_{min} \leq \nu(x) \leq \nu_{max} < \infty$ for all $x \in \mathcal{S}$, then the embedded chain is a discrete time chain (on a general state space $\mathcal{S}$) and it seems that recurrence properties of the embedded discrete time chain would carry over to the continuous time version. The avg number of jumps to a set of states, as measured in the embedded chain, cannot differ too much from the fraction of time being in those states. $\endgroup$ – Michael Oct 8 '16 at 3:18

Another classic in probability theory is William Feller's An Introduction to Probability Theory and its Applications, volume I and II, which has a decent coverage of the topic, perhaps it is worth looking up (more specifically chapter 10, vol. II), even though it is a bit old.


I just finished a course on stochastic calculus, so I still lack experience on continuous time markov processes, but I think I can give you some references I have looked at:

-Chapter 10 of Introduction to Stochastic Integration by Hui-Hsiung Kuo explains the Markov property (and gives examples in terms of the Brownian motion), and diffusion processes. If you have a background in stochastic calculus (this book or Oksendal's are good introductory books) I would start here.

-A more detailed exposition (but too advanced for me at the moment of writing) is the two volume book of Rogers and Williams, Diffussions, Markov Processes and Martingales.

  • $\begingroup$ I know stochastic calculus. I was looking for expositions which did not limit themselves to diffusion processes. $\endgroup$ – Furrer Oct 7 '16 at 9:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.