If I have a Markov chain with finite positive recurrent states $\in S$, then that means starting from a given state $y$, the expected number of steps to return to state $y$ is finite.

Now, if I start at state $j\ne y$ where $y,j\in S$, then I am under the impression that the expected number of steps to reach state $y$ given I start in state $j$ is finite.

I know that state $j$ is positive recurrent too, but I am not sure how to prove the previous statement mathematically?

  • $\begingroup$ I am not sure I know what to expect to return to state y in finitely many steps really means. Can you explain? $\endgroup$ – Did Feb 24 '13 at 19:36
  • $\begingroup$ It means that if I am at state y, then $E_y (T_y)\lt \infty$ where $T_y=min(n\gt 0:X_n=y)$ is the hitting time for state y. I hope that helped. $\endgroup$ – Solver Feb 24 '13 at 19:52
  • $\begingroup$ You might want to modify accordingly your question. $\endgroup$ – Did Feb 24 '13 at 19:56
  • $\begingroup$ They mean the same thing. In the question I am explaining $E_y(T_y)<∞$ in words. $\endgroup$ – Solver Feb 24 '13 at 20:34
  • $\begingroup$ No they do not. Actually I might never have seen the phrase to expect to return to some state in finitely many steps to mean that the expectation of the return time is finite. Note that if one does return to some state, this is always in finitely many steps. $\endgroup$ – Did Feb 24 '13 at 20:55

From the strong Markov property at $T_j$, and the fact that $y$ is positive recurrent we have $$\infty>\mathbb{E}_y(T_y)\geq \mathbb{E}_y(T_y\,1_{(T_j<T_y)}) = \mathbb{E}_y([T_y\circ\theta_{T_j}+T_j]\,1_{(T_j<T_y)})$$ $$\geq \mathbb{E}_y([T_y\circ\theta_{T_j}]\,1_{(T_j<T_y)})=\mathbb{P}_y(T_j<T_y)\,\mathbb{E}_j(T_y). $$ If it is possible to go from state $y$ to state $j$, then $\mathbb{P}_y(T_j<T_y)>0$ and we conclude that $\mathbb{E}_j(T_y)<\infty.$

You must assume that the states communicate. For example, if all states are absorbing then they are all positive recurrent, but $\mathbb{E}_j(T_y)=\infty$ for $j\neq y$.

  • $\begingroup$ For this part: $ℙ_y(T_j\lt T_y)𝔼_j(T_y)=𝔼_y(T_y1_{(T_j\lt T_y)})\lt \infty$. Shouldn't it be: $ℙ_y(T_j\lt T_y)𝔼_j(T_y)\le 𝔼_y(T_y1_{(T_j\lt T_y)})\lt \infty$? $\endgroup$ – Solver Feb 24 '13 at 20:31
  • $\begingroup$ And thanks for the clarification. $\endgroup$ – Solver Feb 24 '13 at 21:25
  • $\begingroup$ @Solver You are right, that should be an inequality. Apologies! $\endgroup$ – user940 Feb 25 '13 at 18:28
  • $\begingroup$ This line of reasoning works for a countably infinite state space, correct? I do not see why $\mathbb{E}_y[T_y] \ge \mathbb{E}_y[T_y 1_{T_j < T_y}]$. Say $\mathbb{E}_y[T_y] = 10$, and state $j$ is at least 20 steps away from state $y$, then is $\mathbb{E}_y[T_y 1_{T_j < T_y}] \ge 20$? Can you explain the notation $[ T_y \circ \theta_{T_j} + T_j ]$ please? Thank you! $\endgroup$ – Ritz Feb 21 '16 at 12:08
  • 1
    $\begingroup$ @Ritz $\theta$ is the shift operator, so $T_j+T_y\circ\theta_{T_j}=\inf(t≥T_j:X_t=y)$, the first time after hitting state $j$ that the process hits state $y$. $\endgroup$ – user940 Feb 21 '16 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.