# Probability limit Markov chain

Let there be an irreducible Markov chain $\{X_t\}_{t\geq 0}$ in continuous time with finitely many possible states. Let $$\displaystyle l_i(T)=\int_{0}^T 1_{(X_t=i)}dt.$$ How can I prove the existence of the limit $$\lim_{T\longrightarrow \infty}\frac{l_i(T)}{T}?$$ This is not a homework, I am studying for an exam.

-

Call $U_0=\inf\{t\geqslant0\mid X_t=i\}$ the first hitting time of $i$ and, for every $n\geqslant1$, $$V_n=\inf\{t\geqslant0\mid X_{U_{n-1}+t}\ne i\}\quad\text{and}\quad U_n=\inf\{t\geqslant0\mid X_{V_n+t}=i\},$$ the duration of the $n$th visit of $i$ and the duration of the $n$th time interval between some consecutive visits to $i$, respectively. Then, all these random variables are independent and integrable, $(U_n)_{n\geqslant1}$ is i.i.d. and $(V_n)_{n\geqslant1}$ is i.i.d. For every $n$, call $$S_n=V_1+\cdots+V_n,\qquad T_n=U_0+U_1+\cdots+U_{n-1}+S_n.$$ For every $n$, if $T_n\leqslant t\lt T_{n+1}$, then $S_n\leqslant\ell_i(t)\lt S_{n+1}$. The law of large numbers applied twice implies that, almost surely, when $n\to\infty$, $$\frac{S_n}n\to v=\mathbb E(V_1),\qquad\frac{T_n}n\to u+v=\mathbb E(U_1)+\mathbb E(V_1),$$ hence, almost surely, when $t\to\infty$, $$\frac{\ell_i(t)}t\to\frac{v}{u+v}.$$ An alternative characterization of the limit is $\pi(i)$, where $\pi$ is the stationary distribution of the Markov process.