Probability limit Markov chain Let there be an irreducible Markov chain $\{X_t\}_{t\geq 0}$ in continuous time with finitely many possible states. 
Let $$\displaystyle l_i(T)=\int_{0}^T 1_{(X_t=i)}dt.$$
How can I prove the existence of the limit 
$$\lim_{T\longrightarrow \infty}\frac{l_i(T)}{T}?$$
This is not a homework, I am studying for an exam.
 A: Look up: "Renewal Theorem"....
A: Call $U_0=\inf\{t\geqslant0\mid X_t=i\}$ the first hitting time of $i$ and, for every $n\geqslant1$, 
$$V_n=\inf\{t\geqslant0\mid X_{U_{n-1}+t}\ne i\}\quad\text{and}\quad 
U_n=\inf\{t\geqslant0\mid X_{V_n+t}=i\},
$$ 
the duration of the $n$th visit of $i$ and the duration of the $n$th time interval between some consecutive visits to $i$, respectively. Then, all these random variables are independent and integrable, $(U_n)_{n\geqslant1}$ is i.i.d. and $(V_n)_{n\geqslant1}$ is i.i.d. For every $n$, call 
$$
S_n=V_1+\cdots+V_n,\qquad
T_n=U_0+U_1+\cdots+U_{n-1}+S_n. 
$$
For every $n$, if $T_n\leqslant t\lt T_{n+1}$, then $S_n\leqslant\ell_i(t)\lt S_{n+1}$. The law of large numbers applied twice implies that, almost surely, when $n\to\infty$,
$$
\frac{S_n}n\to v=\mathbb E(V_1),\qquad\frac{T_n}n\to u+v=\mathbb E(U_1)+\mathbb E(V_1),
$$
hence, almost surely, when $t\to\infty$,
$$
\frac{\ell_i(t)}t\to\frac{v}{u+v}.
$$
An alternative characterization of the limit is $\pi(i)$, where $\pi$ is the stationary distribution of the Markov process.
