# finite state markov chain stationary distribution existence

Here is a fact from Adventures in Stochastic Processes by Prof. Sidney I. Resnick: A finite state, irreducible, aperiodic Markov chain is always positive recurrent and the stationary distribution always exists.

Does this mean that a finite state Markov chain which is irreducible and aperiodic does not have a transient or null recurrent state? Could someone help me prove this?

• Finite, irreducible means you can get to any state in at most size of the state space number of steps with positive probability. Hence... – A.S. Nov 24 '15 at 1:41
• @ A.S.: I think you stated a definition for irreducibility for finite markov chains. I cannot see a rigorous proof from here. – minion Nov 24 '15 at 1:56
• What is a probability that $T_x>N$? What about $T_x>2N$? What about $T_x>kN$? – A.S. Nov 24 '15 at 2:00
• @A.S. The probability $P(T_x > kN) \rightarrow 0$ as $k \rightarrow \infty$ because it is irreducible and finite state. So it would return to $x$ in finite time. – minion Nov 24 '15 at 2:04
• You are left to show that $E(T_x)<\infty$. – A.S. Nov 24 '15 at 2:08