Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider a DTMC with state space $\{1, 2,\ldots {}{} \}$. Now we want to calculate the probability that state 1 is followed by state 2 in the long run i.e, $P(X_n=1, X_{n+1}=2)$ as $n$ tends to infinity. Now if we evaluate this expression then it comes as $\pi_{1}p_{12}$. I have basic conceptual doubt here. If a DTMC reaches state 1 in the long-run then it will stay there for ever. Then the required probability should be zero. I know that I am making a mistake, but not able to find where.

share|cite|improve this question
What makes you think it will stay there forever? It doesn't (necessarily) stay there forever if it starts there, after all. – Jonathan Christensen Jan 11 '13 at 1:24
You should probably try to state rigorously what is meant by the phrase "in the long run". – Nate Eldredge Jan 11 '13 at 1:54
Indeed, if $P(X_n=1)\to\pi(1)$ then $P(X_n=1,X_{n+1}=2)\to\pi(1)p(1,2)$. – Did Jan 11 '13 at 11:13

If a DTMC reaches state 1 in the long-run then it will stay there for ever.

This is vague and, most likely, incorrect. The word "steady" in "steady-state analysis" refers not to the value of $X_n$ being steady, but to the probabilities $P(X_n=k)$, for each fixed $k$. So, the probability of $P(X_n=1)$ may converge to $\pi(1)$ but this does not mean that $X_n$ will stay at $1$. As Did remarked, in this scenario $P(X_n=1,X_{n+1}=2)\to\pi(1)p(1,2)$ as $n\to\infty$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.