0
$\begingroup$

Consider a Continuous-Time Markov Chain (CTMC) with $P_{i,i+1}=1$ and $v_i=i^2$. Here, $P_{i,i+1}$ is the probability of jumping from state $i$ to state $i+1$, and $v_i$ is the rate of holding times. We know a regular CTMC is defined to be a CTMC such that with probability $1$, the number of jumps in any finite time interval is finite. Prove that the above CTMC is not regular. I appreciate any comment/hint.

my attempt: I want to use the fact that the holding time in state $i$, denoted by $\tau_i$ has an exponential distribution with rate $v_i$. Now, consider the interval $[0,t]$. The probability that the number of jumps is finite (say less than a finite number $n$) is equivalent to the probability that that $\sum_{i=1}^n \tau_i > t$. Now, $\sum_{i=1}^n \tau_i$ is the sum of exponential random variables with rates $v_i$. I need to find the pdf of $\sum_{i=1}^n \tau_i$. I do not know how to proceed.

$\endgroup$
3
  • 1
    $\begingroup$ You could show that $\sum_{i=1}^\infty\tau_i<\infty$ with probability 1. One way to do that is to show that $$ \Bbb E\left[\sum_{i=1}^\infty\tau_i\right]<\infty. $$ $\endgroup$ Jan 22, 2021 at 19:18
  • $\begingroup$ Thanks John. If the expectation of a random variable is bounded, it is always true that the random variable itself is also bounded? $\endgroup$
    – Arthur
    Jan 22, 2021 at 20:02
  • 1
    $\begingroup$ If $X$ is a random variable with values in $[0,+\infty]$ and finite expectation $\mu$, then Markov's inequality tells us that $$ \Bbb P[X>t]\le{\mu\over t},\qquad t>0. $$ In particular (let $t\to+\infty$): $\Bbb P[X=+\infty] = 0$, so that $\Bbb P[X<+\infty] = 1$. $\endgroup$ Jan 22, 2021 at 22:37

1 Answer 1

0
$\begingroup$

As suggested by John Dawkins, we can show that $\sum_{i=1}^\infty \tau_i < \infty$. Note that \begin{align} \mathbb{E}[\sum_{i=1}^\infty \tau_i] = \sum_{i=1}^\infty \mathbb{E}[\tau_i] = \sum_{i=1}^\infty \frac{1}{i^2} = \frac{\pi^2}{6} < \infty. \end{align} Since the expectation $\mathbb{E}[\sum_{i=1}^\infty \tau_i] $ is finite, we have $\mathbb{P}(\sum_{i=1}^\infty \tau_i < \infty) = 1$, and this completes the proof.

$\endgroup$
2
  • 1
    $\begingroup$ Careful with your terminology: bounded would mean there was a constant $M>0$ such that $\Bbb P\left[\sum_i\tau_i\le M\right]=1$. The finite-expectation only shows that $\Bbb P\left[\sum_i\tau_i<+\infty\right]=1$. $\endgroup$ Jan 22, 2021 at 22:51
  • $\begingroup$ Thank you for your comment. $\endgroup$
    – Arthur
    Jan 23, 2021 at 1:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .