# Markov chain with finite positive recurrent states

If I have a Markov chain with finite positive recurrent states $\in S$, then that means starting from a given state $y$, the expected number of steps to return to state $y$ is finite.

Now, if I start at state $j\ne y$ where $y,j\in S$, then I am under the impression that the expected number of steps to reach state $y$ given I start in state $j$ is finite.

I know that state $j$ is positive recurrent too, but I am not sure how to prove the previous statement mathematically?

-
I am not sure I know what to expect to return to state y in finitely many steps really means. Can you explain? – Did Feb 24 '13 at 19:36
It means that if I am at state y, then $E_y (T_y)\lt \infty$ where $T_y=min(n\gt 0:X_n=y)$ is the hitting time for state y. I hope that helped. – Solver Feb 24 '13 at 19:52
You might want to modify accordingly your question. – Did Feb 24 '13 at 19:56
They mean the same thing. In the question I am explaining $E_y(T_y)<โ$ in words. – Solver Feb 24 '13 at 20:34
No they do not. Actually I might never have seen the phrase to expect to return to some state in finitely many steps to mean that the expectation of the return time is finite. Note that if one does return to some state, this is always in finitely many steps. – Did Feb 24 '13 at 20:55

From the strong Markov property at $T_j$, and the fact that $y$ is positive recurrent we have $$\infty>\mathbb{E}_y(T_y)\geq \mathbb{E}_y(T_y\,1_{(T_j<T_y)}) = \mathbb{E}_y([T_y\circ\theta_{T_j}+T_j]\,1_{(T_j<T_y)})$$ $$\geq \mathbb{E}_y([T_y\circ\theta_{T_j}]\,1_{(T_j<T_y)})=\mathbb{P}_y(T_j<T_y)\,\mathbb{E}_j(T_y).$$ If it is possible to go from state $y$ to state $j$, then $\mathbb{P}_y(T_j<T_y)>0$ and we conclude that $\mathbb{E}_j(T_y)<\infty.$
You must assume that the states communicate. For example, if all states are absorbing then they are all positive recurrent, but $\mathbb{E}_j(T_y)=\infty$ for $j\neq y$.
For this part: $โ_y(T_j\lt T_y)๐ผ_j(T_y)=๐ผ_y(T_y1_{(T_j\lt T_y)})\lt \infty$. Shouldn't it be: $โ_y(T_j\lt T_y)๐ผ_j(T_y)\le ๐ผ_y(T_y1_{(T_j\lt T_y)})\lt \infty$? – Solver Feb 24 '13 at 20:31
This line of reasoning works for a countably infinite state space, correct? I do not see why $\mathbb{E}_y[T_y] \ge \mathbb{E}_y[T_y 1_{T_j < T_y}]$. Say $\mathbb{E}_y[T_y] = 10$, and state $j$ is at least 20 steps away from state $y$, then is $\mathbb{E}_y[T_y 1_{T_j < T_y}] \ge 20$? Can you explain the notation $[ T_y \circ \theta_{T_j} + T_j ]$ please? Thank you! – Ritz Feb 21 at 12:08
@Ritz $\theta$ is the shift operator, so $T_j+T_y\circ\theta_{T_j}=\inf(tโฅT_j:X_t=y)$, the first time after hitting state $j$ that the process hits state $y$. – Byron Schmuland Feb 21 at 16:22