Here is a fact from Adventures in Stochastic Processes by Prof. Sidney I. Resnick: A finite state, irreducible, aperiodic Markov chain is always positive recurrent and the stationary distribution always exists.

Does this mean that a finite state Markov chain which is irreducible and aperiodic does not have a transient or null recurrent state? Could someone help me prove this?

  • $\begingroup$ Finite, irreducible means you can get to any state in at most size of the state space number of steps with positive probability. Hence... $\endgroup$ – A.S. Nov 24 '15 at 1:41
  • $\begingroup$ @ A.S.: I think you stated a definition for irreducibility for finite markov chains. I cannot see a rigorous proof from here. $\endgroup$ – minion Nov 24 '15 at 1:56
  • $\begingroup$ What is a probability that $T_x>N$? What about $T_x>2N$? What about $T_x>kN$? $\endgroup$ – A.S. Nov 24 '15 at 2:00
  • $\begingroup$ @A.S. The probability $P(T_x > kN) \rightarrow 0$ as $k \rightarrow \infty$ because it is irreducible and finite state. So it would return to $x$ in finite time. $\endgroup$ – minion Nov 24 '15 at 2:04
  • $\begingroup$ You are left to show that $E(T_x)<\infty$. $\endgroup$ – A.S. Nov 24 '15 at 2:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.