Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $\{X_n\}$, $n \geq 0$ be a Markov chain with the transition matrix $P$ such that $$ \begin{array}{c|ccc} &A &B &C \\ \hline A &0.2 & 0.2 &0.6\\ B &0 & 0.25 &0.75\\ C &0.3 & 0.3 & 0.4 \end{array}. $$

How do I find all the stationary distributions?

If the Markov Chain starts in state $A$, what is the expected number of steps before it returns to $A$?

And last one :)

How many times, on average, does the Markov Chain visit state $B$ between two visits to $A$.

I'm finding this topic quite tricky, so I really appreciate the help you guys are giving me :) I'd love if someone could explain all the steps to me.. :)

share|cite|improve this question
These problems are very fundamental in Markov Chains, I think you can find the methods in the textbook. For example, Markov Chains with stationary transition probability(K.L.Chung) – Wayson Kong Mar 30 '13 at 2:10
@Swayy Which textbook(s) are you using? – Did Apr 1 '13 at 21:03
@Did Introduction to Probability, Grinstead – Swayy Apr 14 '13 at 20:27
Surely the author explains how to find the stationary distribution(s) of a Markov chain on 3 states from its transition matrix, no? – Did Apr 14 '13 at 20:38
Introduction to Probability, by Grinstead and Snell? Try section 11.3 Ergodic Markov chains. The theory is in Theorem 11.8, the practice begins with Exemple 11.19 and continues with many other exemples afterwards. – Did Apr 29 '13 at 16:29
up vote 2 down vote accepted

To summarize a discussion in the comments, the OP uses as a textbook Introduction to Probability, by Grinstead and Snell. The authors explain how to find the stationary distribution(s) of a Markov chain on a finite number of states from its transition matrix. Section 11.3 Ergodic Markov chains gives the theory in Theorem 11.8, and explains the practice starting at Exemple 11.19 and continuing with many other exemples afterwards.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.