Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have a Markov chain with finite positive recurrent states $\in S$, then that means starting from a given state $y$, the expected number of steps to return to state $y$ is finite.

Now, if I start at state $j\ne y$ where $y,j\in S$, then I am under the impression that the expected number of steps to reach state $y$ given I start in state $j$ is finite.

I know that state $j$ is positive recurrent too, but I am not sure how to prove the previous statement mathematically?

share|cite|improve this question
I am not sure I know what to expect to return to state y in finitely many steps really means. Can you explain? – Did Feb 24 '13 at 19:36
It means that if I am at state y, then $E_y (T_y)\lt \infty$ where $T_y=min(n\gt 0:X_n=y)$ is the hitting time for state y. I hope that helped. – Solver Feb 24 '13 at 19:52
You might want to modify accordingly your question. – Did Feb 24 '13 at 19:56
They mean the same thing. In the question I am explaining $E_y(T_y)<โˆž$ in words. – Solver Feb 24 '13 at 20:34
No they do not. Actually I might never have seen the phrase to expect to return to some state in finitely many steps to mean that the expectation of the return time is finite. Note that if one does return to some state, this is always in finitely many steps. – Did Feb 24 '13 at 20:55
up vote 2 down vote accepted

From the strong Markov property at $T_j$, and the fact that $y$ is positive recurrent we have $$\infty>\mathbb{E}_y(T_y)\geq \mathbb{E}_y(T_y\,1_{(T_j<T_y)}) = \mathbb{E}_y([T_y\circ\theta_{T_j}+T_j]\,1_{(T_j<T_y)})$$ $$\geq \mathbb{E}_y([T_y\circ\theta_{T_j}]\,1_{(T_j<T_y)})=\mathbb{P}_y(T_j<T_y)\,\mathbb{E}_j(T_y). $$ If it is possible to go from state $y$ to state $j$, then $\mathbb{P}_y(T_j<T_y)>0$ and we conclude that $\mathbb{E}_j(T_y)<\infty.$

You must assume that the states communicate. For example, if all states are absorbing then they are all positive recurrent, but $\mathbb{E}_j(T_y)=\infty$ for $j\neq y$.

share|cite|improve this answer
For this part: $โ„™_y(T_j\lt T_y)๐”ผ_j(T_y)=๐”ผ_y(T_y1_{(T_j\lt T_y)})\lt \infty$. Shouldn't it be: $โ„™_y(T_j\lt T_y)๐”ผ_j(T_y)\le ๐”ผ_y(T_y1_{(T_j\lt T_y)})\lt \infty$? – Solver Feb 24 '13 at 20:31
And thanks for the clarification. – Solver Feb 24 '13 at 21:25
@Solver You are right, that should be an inequality. Apologies! – Byron Schmuland Feb 25 '13 at 18:28
This line of reasoning works for a countably infinite state space, correct? I do not see why $\mathbb{E}_y[T_y] \ge \mathbb{E}_y[T_y 1_{T_j < T_y}]$. Say $\mathbb{E}_y[T_y] = 10$, and state $j$ is at least 20 steps away from state $y$, then is $\mathbb{E}_y[T_y 1_{T_j < T_y}] \ge 20$? Can you explain the notation $[ T_y \circ \theta_{T_j} + T_j ]$ please? Thank you! – Ritz Feb 21 at 12:08
@Ritz $\theta$ is the shift operator, so $T_j+T_y\circ\theta_{T_j}=\inf(tโ‰ฅT_j:X_t=y)$, the first time after hitting state $j$ that the process hits state $y$. – Byron Schmuland Feb 21 at 16:22

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.