Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes me miss the big picture, and I might better look at some other references.

So I would like to know if there are some references with nice treatment on Markov chains/processes? They can be a whole book, some chapters of a book, some lecture notes, some webpages, .... Their approaches can be different, either with or without referring to measure theory (I think a measure-theory approach will be helpful in clarifying things, but I hope it also has some intuitive interpretation), as long as they have clear organization, and provide both a big picture and cover most of the important topics of Markov chains/processes.

I really appreciate your contribution!

share|improve this question
add comment

2 Answers

up vote 5 down vote accepted

Two excellent introductions are James Norris's "Markov Chains" and Pierre Bremaud's "Markov Chains: Gibbs fields, Monte Carlo simulation, and queues". Both books assume a motivated student who is somewhat mathematically mature, though Bremaud reviews basic probability before he gets going.

Also the wonderful book "Markov Chains and Mixing Times" by Levin, Peres, and Wilmer is available online here. It starts right with the definition of Markov Chains, but eventually touches on topics in current research. So it is pretty advanced, but also well worth a look.

share|improve this answer
    
Thanks! I will definitely borrow these books to read. A little question, do you also happen to know some good books on stochastic processes, not specifically for but with relatively concise and nice chapters on Markov chain/processes? –  Ethan Mar 17 '11 at 3:21
    
@Ethan For a gentle introduction that doesn't use measure theory, I quite like Gregory Lawler's "Introduction to Stochastic Processes". He covers both discrete and continuous time Markov processes, and other topics. –  Byron Schmuland Mar 17 '11 at 3:27
    
Thanks! I have some and am now continuing on building my measure theory background. So may I also ask your opinion on some good general stochastic process books with measure theory and nice treatment on Markov processes? –  Ethan Mar 17 '11 at 3:35
1  
@Ethan The two volumes of "Diffusions, Markov Processes, and Martingales" by Rogers and Williams. Now I think you have enough reading to keep you busy for a while! –  Byron Schmuland Mar 17 '11 at 3:41
add comment

While not as advanced as the books mentioned above, if you are looking for examples related to applications of Markov Chains and a nice "brief" treatment you might look at Chapter 5, of Fred Robert's book: Discrete Mathematical Models, Prentice-Hall, 1976.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.