Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a simple CTMC with three states 0,1,2. There are three transitions $0 \rightarrow 1$ (with rate $2u$), $1 \rightarrow 2$ (with rate $u$), $1 \rightarrow 0$ (rate $v$).So $2$ is an absorbing state. Can someone please help me understand how to find the expected time to reach state-2 from state-0? Using the understanding I get, I want to do the analysis on some complex markov chains.

share|cite|improve this question
Did you try the standard method? That is, introduce the expected time t(x) to reach state 2 starting from state x, find a linear system which the t(x) solve, deduce every t(x), and in particular t(0), which is the expected time you are after. – Did Sep 2 '12 at 13:52

Since the OP stays silent, here is an answer. As explained in a comment, the standard method is to introduce the expected time $t_x$ to reach state $2$ starting from each state $x$, to find a linear system which the vector $(t_x)_x$ solve, to deduce $(t_x)_x$, and in particular $t_0$ which is the expected time asked for. Here, the vector is simply $(t_0,t_1)$, and $$t_0=\frac1{2u}+t_1,\qquad t_1=\frac{1}{u+v}+\frac{u}{u+v}\,0+\frac{v}{u+v}t_0,$$ hence $$t_0=\frac1{2u}+\frac1{u+v}+\frac{v}{u+v}t_0,$$ which yields finally $$t_0=\frac{3u+v}{2u^2}.$$

share|cite|improve this answer

You do it in a similar way as you would for finding the expected first passage time for a discrete chain. If $Q = q_{ij}$ is the transition rate matrix, $P = p_{ij}$ is the transition matrix of the embedded chain, \operatorname{Boole} is the indicator function and $T[i, Q]$ is the first passage time starting at state $i$, then

$$\begin{align*} \operatorname{\operatorname{Mean}}[FPT[i, Q]] &= \operatorname{Expectation}[FPT \mid X[0] = i]\\ &= \operatorname{Expectation}[\operatorname{SojournTime} \mid X[0] = i] + \sum _{j=1}^{\infty} p_{ij} \operatorname{Expectation}[FPT \mid X[0] = j]\\ &=-\frac1{q_{ii}} + \sum _{j=1}^{\infty} \frac{-q_{ij}}{q_{ii}} \operatorname{Boole}[j\neq i] \operatorname{Mean}[FPT[j, Q]] \end{align*}$$

where the sojourn/holding time is, of course, exponentially distributed with parameter $-q_{ii}$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.