# Permanence time in a markov chain

For a continuous-time Markov process with, say, two states $$1,2$$ and transition rates $$r_{ik}$$, over a time interval of duration $$T$$, what is the probability $$P(t)$$ of spending, in total, a duration $$t$$ in state $$1$$?

• Do you want a total duration of $t$ perhaps through several visits?
– Ian
Commented Jan 13, 2020 at 12:52

If we begin in state $$1$$ at time $$0$$, then the probability of staying in state $$1$$ at least until time $$t$$ is $$r_{11}^t$$. So, the probability of staying in state $$1$$ until time $$t$$ and then transitioning is $$r_{11}^tr_{12}$$.

Let $$X(t)$$ be the state the system is in at time $$t$$. The generator matrix is given by $$G = \left( \begin{array}{cc} -\lambda & \lambda \\ \mu & -\mu \\ \end{array} \right).$$ From Kolmogorov's backward equation $$P'(t) = GP(t)$$, we find that $$P(t) = e^{Gt}$$, where $$P_{ij}(t) = \mathbb P(X(t)=j\mid X(0)=i)$$. Here $$e^{Gt} = \left( \begin{array}{cc} \frac{\lambda e^{-(\lambda +\mu )t} +\mu }{\lambda +\mu } & \frac{\lambda -\lambda e^{-(\lambda +\mu )t} }{\lambda +\mu } \\ \frac{\mu(1-e^{-(\lambda +\mu )t}) }{\lambda +\mu } & \frac{\lambda +\mu e^{-(\lambda +\mu )t} }{\lambda +\mu } \\ \end{array} \right),$$ so the probability of spending a duration of $$t$$ in state $$1$$ over an interval $$T$$ (assuming $$X(0)=1$$ is given by \begin{align} \frac{\int_0^t P_{11}(s)\ \mathsf ds}{\int_0^T P_{11}(s)\ \mathsf ds} &= \frac{\int_0^t \frac{\lambda e^{-(\lambda +\mu )s} +\mu }{\lambda +\mu }\ \mathsf ds}{\int_0^T \frac{\lambda e^{-(\lambda +\mu )s} +\mu }{\lambda +\mu }\ \mathsf ds}\\ &= \frac{\frac{\lambda +\lambda \left(-e^{-t (\lambda +\mu )}\right)+\mu t (\lambda +\mu )}{(\lambda +\mu )^2}}{\frac{\lambda +\lambda \left(-e^{-T (\lambda +\mu )}\right)+\mu T (\lambda +\mu )}{(\lambda +\mu )^2}}\\ &= \frac{\lambda\left(1-e^{-(\lambda +\mu )t}\right)+\mu (\lambda +\mu )t}{\lambda\left(1-e^{-(\lambda +\mu )T}\right)+\mu (\lambda +\mu )T}. \end{align}