Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Given a simple CTMC with three states 0,1,2. There are three transitions $0 \rightarrow 1$ (with rate $2u$), $1 \rightarrow 2$ (with rate $u$), $1 \rightarrow 0$ (rate $v$).So $2$ is an absorbing state. Can someone please help me understand how to find the expected time to reach state-2 from state-0? Using the understanding I get, I want to do the analysis on some complex markov chains.

share|improve this question
Did you try the standard method? That is, introduce the expected time t(x) to reach state 2 starting from state x, find a linear system which the t(x) solve, deduce every t(x), and in particular t(0), which is the expected time you are after. –  Did Sep 2 '12 at 13:52
add comment

2 Answers

You do it in a similar way as you would for finding the expected first passage time for a discrete chain. If $Q = q_{ij}$ is the transition rate matrix, $P = p_{ij}$ is the transition matrix of the embedded chain, \operatorname{Boole} is the indicator function and $T[i, Q]$ is the first passage time starting at state $i$, then

$$\begin{align*} \operatorname{\operatorname{Mean}}[FPT[i, Q]] &= \operatorname{Expectation}[FPT \mid X[0] = i]\\ &= \operatorname{Expectation}[\operatorname{SojournTime} \mid X[0] = i] + \sum _{j=1}^{\infty} p_{ij} \operatorname{Expectation}[FPT \mid X[0] = j]\\ &=-\frac1{q_{ii}} + \sum _{j=1}^{\infty} \frac{-q_{ij}}{q_{ii}} \operatorname{Boole}[j\neq i] \operatorname{Mean}[FPT[j, Q]] \end{align*}$$

where the sojourn/holding time is, of course, exponentially distributed with parameter $-q_{ii}$.

share|improve this answer
add comment

Since the OP stays silent, here is an answer. As explained in a comment, the standard method is to introduce the expected time $t_x$ to reach state $2$ starting from each state $x$, to find a linear system which the vector $(t_x)_x$ solve, to deduce $(t_x)_x$, and in particular $t_0$ which is the expected time asked for. Here, $$t_0=\frac1{2u}+t_1,\qquad t_1=\frac{v}{u+v}\left(\frac1v+t_0\right)+\frac{u}{u+v}\frac1u,$$ hence $$t_0=\frac1{2u}+\frac2{u+v}+\frac{v}{u+v}t_0,$$ which yields finally $$t_0=\frac{5u+v}{2u^2}.$$

share|improve this answer
add comment

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.