I have some questions regarding CTMC... and most importantly whether the step-by-step example I provide below is correct. My main sources about CTMC are: ([1], and [2]).
Let's assume 3 possible states $S = {1, 2, 3}$. There are the following data instances $D = {d_1, d_2, d_3, d_4}$ from which I want to model as a CTMC. In parenthesis the time spent at the state before transiting to the next.
$ d_1 = 1(3min) \rightarrow 2(3min) \rightarrow 3\\ d_2 = 1(8min) \rightarrow 2(12min) \rightarrow 3\\ d_3 = 1(2min) \rightarrow 3\\ d_4 = 2(2min) \rightarrow 3 $
The main question(s) I want to ask and answer from the CTMC is:
- Problem: What is the probability of transiting from state $i$ to state $j$ given that $s$ time has elapsed since entering state $i$?
So to form the CTMC I start with the embedded transition matrix for $D$ which is:
$ p_{ij} = \begin{bmatrix} 0 & 2/3 & 1/3 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \end{bmatrix} $
Now the next step is to calculate the rates. From [1, p3] it says that "Each time state i is visited, the chain spends, on average, $E(Hi) = 1/a_i$ units of time there before moving on". From what I understand $a_i$ is the set of rates for state $i$, also denoted in other texts as $\lambda_i$ or $v_i$, for example an operator receives 2 calls in an hour, hence $a_i = 1$ call in half an hour. However, I am not sure how to calculate this from the above data $D$. But, I can calculate the average time spent at each state and then from the above equation, solving for $a_i = 1 / E(H_i)$ I can obtain these rates...
$E(H_i) = \begin{bmatrix}4.33 & 5.66 & 0 \end{bmatrix}$, denoting with 0 the terminal (consume) state.
Hence,
$a_i = \begin{bmatrix}0.23 & 0.18 & 0 \end{bmatrix}$, again 0 is for consume state.
- Q1: Is this valid?
- Q2: Given that in this way I assume that the probability distribution of time spent is an exponential distribution, I lose the true variance of the actual distribution as the variance of the exponential distribution is the same as the mean... Is this OK? Is there any other way?
- Q3: How can terminal/consume states be represented?
Again from [1, p3] I can now compute the transition rate matrix (infinitesimal generator) $Q$:
$Q_{ij} = -a_i$, if $i==j$
$Q_{ij} = a_i * p_{ij}$, if $i != j$
$ Q = \begin{bmatrix} -0.23 & 0.15 & 0.08\\ 0 & -0.18 & 0.18\\ 0 & 0 & 0 \end{bmatrix} $
- Q4: Although [1, p3] states that $Q_{ii} = -a_i$, [2, p227] says that $Q_{ii} = 0$. Which one is correct then?
Now that we know $Q$, from [1, p2] I can calculate $P_{ij}(t)$ as follows:
$P_{ij}(t) = e^{Qt}$ (from what I read there are a number of ways to compute that and I can use a numerical software that implements them)
- Q5: I assume $t = s$, ie the elapsed time since entering state $i$
- Q6: Why do I need to calculate the complete matrix $P_{ij}(t)$ if I am interested in one particular ${ij}$. Is there any way to calculate that.
- Q7: I read in [2, p.228] that if the process can be modelled as a Poisson process then the solution is: $P_{ij}(t) = \dfrac{(\lambda_i t)^k}{k!}e^{-\lambda_i t}$, where $k$ is the number of events in an interval of length t, which in my case above is $1$. Can I assume that the above process is a Poisson process?
- Q8: And lastly is everything I have explained in the above step-by-step example correct?
Thank you the most for your time reading this and if you can provide any answers to my questions.