I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes me miss the big picture, and I might better look at some other references.

So I would like to know if there are some references with nice treatment on Markov chains/processes? They can be a whole book, some chapters of a book, some lecture notes, some webpages, .... Their approaches can be different, either with or without referring to measure theory (I think a measure-theory approach will be helpful in clarifying things, but I hope it also has some intuitive interpretation), as long as they have clear organization, and provide both a big picture and cover most of the important topics of Markov chains/processes.

I really appreciate your contribution!


Two excellent introductions are James Norris's "Markov Chains" and Pierre Bremaud's "Markov Chains: Gibbs fields, Monte Carlo simulation, and queues". Both books assume a motivated student who is somewhat mathematically mature, though Bremaud reviews basic probability before he gets going.

Also the wonderful book "Markov Chains and Mixing Times" by Levin, Peres, and Wilmer is available online here. It starts right with the definition of Markov Chains, but eventually touches on topics in current research. So it is pretty advanced, but also well worth a look.

  • $\begingroup$ Thanks! I will definitely borrow these books to read. A little question, do you also happen to know some good books on stochastic processes, not specifically for but with relatively concise and nice chapters on Markov chain/processes? $\endgroup$ – Ethan Mar 17 '11 at 3:21
  • 1
    $\begingroup$ @Ethan For a gentle introduction that doesn't use measure theory, I quite like Gregory Lawler's "Introduction to Stochastic Processes". He covers both discrete and continuous time Markov processes, and other topics. $\endgroup$ – user940 Mar 17 '11 at 3:27
  • $\begingroup$ Thanks! I have some and am now continuing on building my measure theory background. So may I also ask your opinion on some good general stochastic process books with measure theory and nice treatment on Markov processes? $\endgroup$ – Ethan Mar 17 '11 at 3:35
  • 2
    $\begingroup$ @Ethan The two volumes of "Diffusions, Markov Processes, and Martingales" by Rogers and Williams. Now I think you have enough reading to keep you busy for a while! $\endgroup$ – user940 Mar 17 '11 at 3:41

While not as advanced as the books mentioned above, if you are looking for examples related to applications of Markov Chains and a nice "brief" treatment you might look at Chapter 5, of Fred Robert's book: Discrete Mathematical Models, Prentice-Hall, 1976.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.