Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let there be an irreducible Markov chain $\{X_t\}_{t\geq 0}$ in continuous time with finitely many possible states. Let $$\displaystyle l_i(T)=\int_{0}^T 1_{(X_t=i)}dt.$$ How can I prove the existence of the limit $$\lim_{T\longrightarrow \infty}\frac{l_i(T)}{T}?$$ This is not a homework, I am studying for an exam.

share|cite|improve this question

Look up: "Renewal Theorem"....

share|cite|improve this answer

Call $U_0=\inf\{t\geqslant0\mid X_t=i\}$ the first hitting time of $i$ and, for every $n\geqslant1$, $$V_n=\inf\{t\geqslant0\mid X_{U_{n-1}+t}\ne i\}\quad\text{and}\quad U_n=\inf\{t\geqslant0\mid X_{V_n+t}=i\}, $$ the duration of the $n$th visit of $i$ and the duration of the $n$th time interval between some consecutive visits to $i$, respectively. Then, all these random variables are independent and integrable, $(U_n)_{n\geqslant1}$ is i.i.d. and $(V_n)_{n\geqslant1}$ is i.i.d. For every $n$, call $$ S_n=V_1+\cdots+V_n,\qquad T_n=U_0+U_1+\cdots+U_{n-1}+S_n. $$ For every $n$, if $T_n\leqslant t\lt T_{n+1}$, then $S_n\leqslant\ell_i(t)\lt S_{n+1}$. The law of large numbers applied twice implies that, almost surely, when $n\to\infty$, $$ \frac{S_n}n\to v=\mathbb E(V_1),\qquad\frac{T_n}n\to u+v=\mathbb E(U_1)+\mathbb E(V_1), $$ hence, almost surely, when $t\to\infty$, $$ \frac{\ell_i(t)}t\to\frac{v}{u+v}. $$ An alternative characterization of the limit is $\pi(i)$, where $\pi$ is the stationary distribution of the Markov process.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.