Markov chain basic positive recurrency question

If a discrete markov chain is stationary (as far as I know: doesn't modify itself with time), irreducible (doesn't have transient states) and aperiodic (no periodic states), is it positive recurrent?

This answer might be answered or not, the problem is: I don't know whether it can.

A chain is positive recurrent if mean recurrence time is finite, but it seems to me that I don't know how many states are there in a generic discrete markov chain (could it be infinite states?).

• Irreducibility is not defined as having no transient state. – Did May 17 '12 at 15:24
• You're right, I meant "every state can communicate with any other state" (sorry I lost the account pw) – Marco A. May 17 '12 at 15:36

If by "stationary" you mean that the transition probabilities $P(X_{n+1} = y \mid X_n = x)$ are independent of $n$ (also called "time homogeneous"), then the answer is certainly no, if the chain can have an infinite number of states. Consider a chain on the integers where at each step you move one unit to the right with probability 1 (i.e. $P(X_{n+1} = x+1 \mid X_n = x) =1$); then every state is transient.

If by "stationary" you mean the chain admits a stationary distribution, i.e. there is a probability measure $\pi$ such that $P_\pi(X_n = x) = \pi(x)$ for every $n,x$, then the answer is yes, every state is positive recurrent, and indeed the expected return time $E_x T_x$ is given by $1/\pi(x)$. You do not need aperiodicity. This is a standard fact and should be in most textbooks.

A chain with a finite number of states always admits a stationary distribution, so in the first example, having an infinite number of states was necessary.

• Thank you, I meant the first case. So the answer is "no" because the chain might have an infinite number of states – Marco A. May 17 '12 at 15:39
• Actually this example is wrong because the chain is not irreducible (in the usual sense of the word). Asymmetric random walk would be a better example. If you want to avoid transient states then use symmetric random walk. If you want to avoid periodicity then make it "lazy" by giving the chain a positive probability of not moving. – Nate Eldredge May 17 '12 at 16:52

Yes a Markov chain could have an infinite number of states and so I think the best you can say is that the chain is null recurrent meaning every state with be returned to but the waiting time to return has no finite bound.

• Sorry but a Markov chain with an infinite number of states may be (irreducible and) positive recurrent. – Did May 17 '12 at 15:23
• @Didier I wasn't saying that all infinite state Markov chains were null recurrent. What I was saying is that by virtue of the chain having infinte number of states you couldn't rule out that it was null recurrent. You obviously are not saying that no infinite state Markov Chains can be null recurrent. – Michael Chernick May 17 '12 at 17:09