Tell me more ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.
up vote 0 down vote favorite

Consider a DTMC with state space $\{1, 2,\ldots {}{} \}$. Now we want to calculate the probability that state 1 is followed by state 2 in the long run i.e, $P(X_n=1, X_{n+1}=2)$ as $n$ tends to infinity. Now if we evaluate this expression then it comes as $\pi_{1}p_{12}$. I have basic conceptual doubt here. If a DTMC reaches state 1 in the long-run then it will stay there for ever. Then the required probability should be zero. I know that I am making a mistake, but not able to find where.

share|improve this question
1  
What makes you think it will stay there forever? It doesn't (necessarily) stay there forever if it starts there, after all. – Jonathan Christensen Jan 11 at 1:24
1  
You should probably try to state rigorously what is meant by the phrase "in the long run". – Nate Eldredge Jan 11 at 1:54
Indeed, if $P(X_n=1)\to\pi(1)$ then $P(X_n=1,X_{n+1}=2)\to\pi(1)p(1,2)$. – Did Jan 11 at 11:13
0% accept rate? – Did Jan 11 at 11:13

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.