Consider $(X_n)$ a discrete-time Markov chain with values in a countable space $E$. Let $x$ be a transient state, let $R$ denote the set of recurrent states and $P_x$ the probability measure when starting at $x$. Define $T=\inf\{n\geq 0, X_n \in R\}$.

Is it true that either $P_x(T=\infty)=1$ or $P_x(T<\infty)=1$ ?

I believe this has been claimed in some lecture notes I'm reading. However, no proof is given.

I'd consider two cases: whether $x$ communicates with some recurrent $y$ or not. If there is no such $y$, then $T$ is always $\infty$. On the contrary, if $x$ communicates with some recurrent $y$, I don't think $y$ is necessarily hit with probability $1$, thus there is no reason to believe that $P_x(T<\infty)=1$.


Consider a random walk on $\Bbb N_0$ with transition probabilities as follows. $p_{o,o}=1,p_{1,0}=1/2=p_{1,2},p_{n,n+1}=1$ for all $n>1$. Then $0$ is the only recurrent state, and the hitting time of $0$ starting from $1$ is $1$ with probability $1/2$ and $\infty$ with probability $1/2$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.