# Reference request for this topics

I'm a second year undergraduate statistic student, I need a good reference to learn these topics

1. Markov Chains in discrete time.

1.1. Classification of states, recurrence notions of transience.

1.2. Stationary measure.

1.3. Reversibility.

2. Markov Chains continuous time.

3. Poisson process.
4. Processes of birth and death.
5. Applications: renewal theory notions of queuing theory.

I've had a look and seem to have video classes at MIT, I appreciate any indication

Is there any significant difference between:

1. Sheldon M. Ross "Stochastic processes"
2. Sheldon M. Ross "Introduction to probability models"

EDIT:I'm sorry for reviving this question, but I've been trying to study through 2. Sheldon M. Ross "Introduction to probability models", but this book is very tiring to read, now I'm looking for lecture notes or a more concise book only for the theory without many examples, and use Ross's book only to solve the exercises, since this has solutions.

Has anyone used any of the books below for reference?

Essentials of Stochastic Processes

A First Course in Stochastic Processes

• Firstly, I know the book of Durrett and can confirm that there are several typos but otherwise it is a nicely written book. One thing you should consider is, that it might be frustrating to separate the primary book and the exercises/problems/examples due to notation differences and knowledge requirement. Did you give the first book (Suhov,Kelbert) I recommend a look? Commented Jul 8, 2015 at 20:10
• @user190080 What's the name of the book? Commented Jul 8, 2015 at 20:12
• "Probability and Statistics by Example: Volume 2, Markov Chains: A Primer in Random Processes and their Applications" just like in the answer :) I really like it Commented Jul 8, 2015 at 20:13
• @user190080 Was trying to find this on the internet as in the library of my university does not, I'll keep trying to find it Commented Jul 8, 2015 at 20:22

A good starting point would be Introduction to Probability Models by Sheldon Ross. I think it covers all the topics that you described. I like the book because it's easy to read and has plenty of problems to try out, which makes it ideal for self learning.

• Agree. Sheldon M. Ross "Stochastic processes" is a little more advanced. Commented Jul 2, 2015 at 19:04
• @PeterR I will take a look first at Introduction to Probability Models then. Commented Jul 2, 2015 at 19:37

I can recommend the following book

"Probability and Statistics by Example: Volume 2, Markov Chains: A Primer in Random Processes and their Applications" by Yuri Suhov and Mark Kelbert

This book covers most of your topics (depends of course how deep you want to dive into), is well written and explains everything with examples and solved exercises. So at least definitely a good source for exercises.

A very good companion for further readings in Markov chains on general state spaces is this book

"S.P. Meyn and R.L. Tweedie (1993), Markov chains and stochastic stability"