# Stopping time for irreducible recurrent finite Markov chains in continuous time

Let $$(X_t)_t$$ be a continuous-time Markov chain with values in a finite set $$E$$.

Assume that $$\forall x,y \in E, \forall t > 0, \quad \mathbb{P}(X_t = y | X_0 = x) > 0$$.

How do you prove that $$\forall x,y \in E, \quad \mathbb{E}[\inf \{ t > 0, X_t = y \} | X_0 = x ] < +\infty$$?

The method I found consists in considering the discrete-time Markov chain $$(X_n)_{n \in \mathbb{N}}$$ but there should be a continuous-time method.

I write $$\Bbb E^x$$ and $$\Bbb P^x$$ to indicate the initial condition $$X_0=x$$.
Define $$u(x):=\Bbb E^x\left[\int_0^\infty e^{-t} 1_{\{X_t=y\}}dt\right]$$. Then $$\delta:=\inf\{u(x):x\in E\}>0$$. Choose $$b>0$$ so large that $$e^{-b}<\delta/2$$. Writing $$T_y:=\inf\{t>0:X_t=y\}$$, you have \eqalign{ v(x) &\le\Bbb E^x\left[\int_0^b e^{-t}1_{\{X_t=y\}}dt\right]+e^{-b}\cr &=\Bbb E^x\left[\int_0^b e^{-t}1_{\{X_t=y\}}1_{\{T_y\le b\}}dt\right]+e^{-b}\cr &\le\Bbb E^x\left[\int_0^b e^{-t}1_{\{T_y\le b\}}dt\right]+e^{-b}\cr &=(1-e^{-b})\Bbb P^x\left[T_y\le b\right]+e^{-b}\cr &\le\Bbb P^x\left[T_y\le b\right]+e^{-b}\cr } It follows that $$\Bbb P^x[T_y>b]\le 1-\delta/2,\qquad\forall x.$$ Noting that $$\{T_y>2b\} =\{T_y>b\}\cap\{\inf\{t>b:X_t=y\}>b\}$$, and using the Markov property at time $$b$$: \eqalign{ \Bbb P^x[T_y>2b] &=\Bbb E^x\left[ 1_{\{T_y>b\}}\Bbb P^{X_b}[T_y>b]\right]\cr &\le\Bbb E^x\left[ 1_{\{T_y>b\}}\cdot(1-\delta/2)\right]\cr &\le(1-\delta/2)^2. } Repeating this you see that $$\Bbb P^x[T_y>nb]\le(1-\delta/2)^n,\qquad n=1,2,\ldots, x\in E.$$ It follows easily from this that there is a constant $$C>0$$ such that $$\Bbb P^x[T_y>t]\le Ce^{-\rho t},\qquad t>0, x\in E,$$ where $$\rho:=-[\log(1-\delta/2)]/b$$. This implies that $$T_y$$ has finite moments of all orders; indeed $$\Bbb E^x[\exp(\alpha T_y)]<\infty,\qquad x\in E,$$ provided $$\alpha<\rho$$.