Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

If I have a Markov chain with finite positive recurrent states $\in S$, then that means starting from a given state $y$, the expected number of steps to return to state $y$ is finite.

Now, if I start at state $j\ne y$ where $y,j\in S$, then I am under the impression that the expected number of steps to reach state $y$ given I start in state $j$ is finite.

I know that state $j$ is positive recurrent too, but I am not sure how to prove the previous statement mathematically?

share|improve this question
    
I am not sure I know what to expect to return to state y in finitely many steps really means. Can you explain? –  Did Feb 24 '13 at 19:36
    
It means that if I am at state y, then $E_y (T_y)\lt \infty$ where $T_y=min(n\gt 0:X_n=y)$ is the hitting time for state y. I hope that helped. –  Solver Feb 24 '13 at 19:52
    
You might want to modify accordingly your question. –  Did Feb 24 '13 at 19:56
    
They mean the same thing. In the question I am explaining $E_y(T_y)<โˆž$ in words. –  Solver Feb 24 '13 at 20:34
    
No they do not. Actually I might never have seen the phrase to expect to return to some state in finitely many steps to mean that the expectation of the return time is finite. Note that if one does return to some state, this is always in finitely many steps. –  Did Feb 24 '13 at 20:55

1 Answer 1

up vote 1 down vote accepted

From the strong Markov property, and the fact that $y$ is positive recurrent we have $$\mathbb{P}_y(T_j<T_y)\,\mathbb{E}_j(T_y)\leq\mathbb{E}_y(T_y\,1_{(T_j<T_y)})<\infty.$$ If it is possible to go from state $y$ to state $j$, then $\mathbb{P}_y(T_j<T_y)>0$ and we conclude that $\mathbb{E}_j(T_y)<\infty.$

You must assume that the states communicate. For example, if all states are absorbing then they are all positive recurrent, but $\mathbb{E}_j(T_y)=\infty$ for $j\neq y$.

share|improve this answer
    
For this part: $โ„™_y(T_j\lt T_y)๐”ผ_j(T_y)=๐”ผ_y(T_y1_{(T_j\lt T_y)})\lt \infty$. Shouldn't it be: $โ„™_y(T_j\lt T_y)๐”ผ_j(T_y)\le ๐”ผ_y(T_y1_{(T_j\lt T_y)})\lt \infty$? –  Solver Feb 24 '13 at 20:31
    
And thanks for the clarification. –  Solver Feb 24 '13 at 21:25
    
@Solver You are right, that should be an inequality. Apologies! –  Byron Schmuland Feb 25 '13 at 18:28

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.