I'm a second year undergraduate statistic student, I need a good reference to learn these topics

  1. Markov Chains in discrete time.

       1.1. Classification of states, recurrence notions of transience.

       1.2. Stationary measure.

       1.3. Reversibility.

  2. Markov Chains continuous time.

  3. Poisson process.
  4. Processes of birth and death.
  5. Applications: renewal theory notions of queuing theory.

I've had a look and seem to have video classes at MIT, I appreciate any indication

Is there any significant difference between:

  1. Sheldon M. Ross "Stochastic processes"
  2. Sheldon M. Ross "Introduction to probability models"

EDIT:I'm sorry for reviving this question, but I've been trying to study through 2. Sheldon M. Ross "Introduction to probability models", but this book is very tiring to read, now I'm looking for lecture notes or a more concise book only for the theory without many examples, and use Ross's book only to solve the exercises, since this has solutions.

Has anyone used any of the books below for reference?

Essentials of Stochastic Processes

A First Course in Stochastic Processes

  • 1
    $\begingroup$ Firstly, I know the book of Durrett and can confirm that there are several typos but otherwise it is a nicely written book. One thing you should consider is, that it might be frustrating to separate the primary book and the exercises/problems/examples due to notation differences and knowledge requirement. Did you give the first book (Suhov,Kelbert) I recommend a look? $\endgroup$
    – user190080
    Commented Jul 8, 2015 at 20:10
  • $\begingroup$ @user190080 What's the name of the book? $\endgroup$
    – Roland
    Commented Jul 8, 2015 at 20:12
  • $\begingroup$ "Probability and Statistics by Example: Volume 2, Markov Chains: A Primer in Random Processes and their Applications" just like in the answer :) I really like it $\endgroup$
    – user190080
    Commented Jul 8, 2015 at 20:13
  • $\begingroup$ @user190080 Was trying to find this on the internet as in the library of my university does not, I'll keep trying to find it $\endgroup$
    – Roland
    Commented Jul 8, 2015 at 20:22

2 Answers 2


A good starting point would be Introduction to Probability Models by Sheldon Ross. I think it covers all the topics that you described. I like the book because it's easy to read and has plenty of problems to try out, which makes it ideal for self learning.

  • 1
    $\begingroup$ Agree. Sheldon M. Ross "Stochastic processes" is a little more advanced. $\endgroup$
    – PeterR
    Commented Jul 2, 2015 at 19:04
  • $\begingroup$ @PeterR I will take a look first at Introduction to Probability Models then. $\endgroup$
    – Roland
    Commented Jul 2, 2015 at 19:37

I can recommend the following book

"Probability and Statistics by Example: Volume 2, Markov Chains: A Primer in Random Processes and their Applications" by Yuri Suhov and Mark Kelbert

This book covers most of your topics (depends of course how deep you want to dive into), is well written and explains everything with examples and solved exercises. So at least definitely a good source for exercises.

A very good companion for further readings in Markov chains on general state spaces is this book

"S.P. Meyn and R.L. Tweedie (1993), Markov chains and stochastic stability"

you can access it here for free.

I wouldn't recommend it for you as a primary source since it doesn't match exactly your criteria, but it is a really well written book, a lot of motivational examples and covers quite some theory.



You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .