Stopping time for irreducible recurrent finite Markov chains in continuous time

Question

Let $(X_t)_t$ be a continuous-time Markov chain with values in a finite set $E$.

Assume that $\forall x,y \in E, \forall t > 0, \quad \mathbb{P}(X_t = y | X_0 = x) > 0$.

How do you prove that $\forall x,y \in E, \quad \mathbb{E}[\inf \{ t > 0, X_t = y \} | X_0 = x ] < +\infty$?

The method I found consists in considering the discrete-time Markov chain $(X_n)_{n \in \mathbb{N}}$ but there should be a continuous-time method.

Answer 1

I write $\Bbb E^x$ and $\Bbb P^x$ to indicate the initial condition $X_0=x$.

Define $u(x):=\Bbb E^x\left[\int_0^\infty e^{-t} 1_{\{X_t=y\}}dt\right]$. Then $\delta:=\inf\{u(x):x\in E\}>0$. Choose $b>0$ so large that $e^{-b}<\delta/2$. Writing $T_y:=\inf\{t>0:X_t=y\}$, you have $$ \eqalign{ v(x) &\le\Bbb E^x\left[\int_0^b e^{-t}1_{\{X_t=y\}}dt\right]+e^{-b}\cr &=\Bbb E^x\left[\int_0^b e^{-t}1_{\{X_t=y\}}1_{\{T_y\le b\}}dt\right]+e^{-b}\cr &\le\Bbb E^x\left[\int_0^b e^{-t}1_{\{T_y\le b\}}dt\right]+e^{-b}\cr &=(1-e^{-b})\Bbb P^x\left[T_y\le b\right]+e^{-b}\cr &\le\Bbb P^x\left[T_y\le b\right]+e^{-b}\cr } $$ It follows that $$ \Bbb P^x[T_y>b]\le 1-\delta/2,\qquad\forall x. $$ Noting that $\{T_y>2b\} =\{T_y>b\}\cap\{\inf\{t>b:X_t=y\}>b\}$, and using the Markov property at time $b$: $$ \eqalign{ \Bbb P^x[T_y>2b] &=\Bbb E^x\left[ 1_{\{T_y>b\}}\Bbb P^{X_b}[T_y>b]\right]\cr &\le\Bbb E^x\left[ 1_{\{T_y>b\}}\cdot(1-\delta/2)\right]\cr &\le(1-\delta/2)^2. } $$ Repeating this you see that $$ \Bbb P^x[T_y>nb]\le(1-\delta/2)^n,\qquad n=1,2,\ldots, x\in E. $$ It follows easily from this that there is a constant $C>0$ such that $$ \Bbb P^x[T_y>t]\le Ce^{-\rho t},\qquad t>0, x\in E, $$ where $\rho:=-[\log(1-\delta/2)]/b$. This implies that $T_y$ has finite moments of all orders; indeed $$ \Bbb E^x[\exp(\alpha T_y)]<\infty,\qquad x\in E, $$ provided $\alpha<\rho$.