1
$\begingroup$

For a continuous-time Markov process with, say, two states $1,2$ and transition rates $r_{ik}$, over a time interval of duration $T$, what is the probability $P(t)$ of spending, in total, a duration $t$ in state $1$?

$\endgroup$
1
  • $\begingroup$ Do you want a total duration of $t$ perhaps through several visits? $\endgroup$
    – Ian
    Commented Jan 13, 2020 at 12:52

2 Answers 2

0
$\begingroup$

If we begin in state $1$ at time $0$, then the probability of staying in state $1$ at least until time $t$ is $r_{11}^t$. So, the probability of staying in state $1$ until time $t$ and then transitioning is $r_{11}^tr_{12}$.

$\endgroup$
0
$\begingroup$

Let $X(t)$ be the state the system is in at time $t$. The generator matrix is given by $$ G = \left( \begin{array}{cc} -\lambda & \lambda \\ \mu & -\mu \\ \end{array} \right). $$ From Kolmogorov's backward equation $P'(t) = GP(t)$, we find that $P(t) = e^{Gt}$, where $P_{ij}(t) = \mathbb P(X(t)=j\mid X(0)=i)$. Here $$ e^{Gt} = \left( \begin{array}{cc} \frac{\lambda e^{-(\lambda +\mu )t} +\mu }{\lambda +\mu } & \frac{\lambda -\lambda e^{-(\lambda +\mu )t} }{\lambda +\mu } \\ \frac{\mu(1-e^{-(\lambda +\mu )t}) }{\lambda +\mu } & \frac{\lambda +\mu e^{-(\lambda +\mu )t} }{\lambda +\mu } \\ \end{array} \right), $$ so the probability of spending a duration of $t$ in state $1$ over an interval $T$ (assuming $X(0)=1$ is given by \begin{align} \frac{\int_0^t P_{11}(s)\ \mathsf ds}{\int_0^T P_{11}(s)\ \mathsf ds} &= \frac{\int_0^t \frac{\lambda e^{-(\lambda +\mu )s} +\mu }{\lambda +\mu }\ \mathsf ds}{\int_0^T \frac{\lambda e^{-(\lambda +\mu )s} +\mu }{\lambda +\mu }\ \mathsf ds}\\ &= \frac{\frac{\lambda +\lambda \left(-e^{-t (\lambda +\mu )}\right)+\mu t (\lambda +\mu )}{(\lambda +\mu )^2}}{\frac{\lambda +\lambda \left(-e^{-T (\lambda +\mu )}\right)+\mu T (\lambda +\mu )}{(\lambda +\mu )^2}}\\ &= \frac{\lambda\left(1-e^{-(\lambda +\mu )t}\right)+\mu (\lambda +\mu )t}{\lambda\left(1-e^{-(\lambda +\mu )T}\right)+\mu (\lambda +\mu )T}. \end{align}

$\endgroup$

You must log in to answer this question.