# Non-regular Continuous-Time Markov Chain

Consider a Continuous-Time Markov Chain (CTMC) with $$P_{i,i+1}=1$$ and $$v_i=i^2$$. Here, $$P_{i,i+1}$$ is the probability of jumping from state $$i$$ to state $$i+1$$, and $$v_i$$ is the rate of holding times. We know a regular CTMC is defined to be a CTMC such that with probability $$1$$, the number of jumps in any finite time interval is finite. Prove that the above CTMC is not regular. I appreciate any comment/hint.

my attempt: I want to use the fact that the holding time in state $$i$$, denoted by $$\tau_i$$ has an exponential distribution with rate $$v_i$$. Now, consider the interval $$[0,t]$$. The probability that the number of jumps is finite (say less than a finite number $$n$$) is equivalent to the probability that that $$\sum_{i=1}^n \tau_i > t$$. Now, $$\sum_{i=1}^n \tau_i$$ is the sum of exponential random variables with rates $$v_i$$. I need to find the pdf of $$\sum_{i=1}^n \tau_i$$. I do not know how to proceed.

• You could show that $\sum_{i=1}^\infty\tau_i<\infty$ with probability 1. One way to do that is to show that $$\Bbb E\left[\sum_{i=1}^\infty\tau_i\right]<\infty.$$ Jan 22, 2021 at 19:18
• Thanks John. If the expectation of a random variable is bounded, it is always true that the random variable itself is also bounded? Jan 22, 2021 at 20:02
• If $X$ is a random variable with values in $[0,+\infty]$ and finite expectation $\mu$, then Markov's inequality tells us that $$\Bbb P[X>t]\le{\mu\over t},\qquad t>0.$$ In particular (let $t\to+\infty$): $\Bbb P[X=+\infty] = 0$, so that $\Bbb P[X<+\infty] = 1$. Jan 22, 2021 at 22:37

As suggested by John Dawkins, we can show that $$\sum_{i=1}^\infty \tau_i < \infty$$. Note that \begin{align} \mathbb{E}[\sum_{i=1}^\infty \tau_i] = \sum_{i=1}^\infty \mathbb{E}[\tau_i] = \sum_{i=1}^\infty \frac{1}{i^2} = \frac{\pi^2}{6} < \infty. \end{align} Since the expectation $$\mathbb{E}[\sum_{i=1}^\infty \tau_i]$$ is finite, we have $$\mathbb{P}(\sum_{i=1}^\infty \tau_i < \infty) = 1$$, and this completes the proof.
• Careful with your terminology: bounded would mean there was a constant $M>0$ such that $\Bbb P\left[\sum_i\tau_i\le M\right]=1$. The finite-expectation only shows that $\Bbb P\left[\sum_i\tau_i<+\infty\right]=1$. Jan 22, 2021 at 22:51