Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Given a markov process $\mathcal{Y} = (Y_t : t \ge 0)$ with state space $E=\{1,2,3\}$ and with generator matrix

$G = \left[ \begin{array}{ccc} -3 & 1 & 2 \\ 1 & -2 & 1 \\ 0 & 0 & 0 \end{array} \right]$.

Assume that the initial distribution is given by $\alpha_1 = \mathbb{P}(Y_0 =1) = 3/4$ and $\alpha_2 = \mathbb{P}(Y_0 =2) = 1/4$.

How do I find the distribution of the first hitting time $Z: = \min\{t \ge 0 : Y_t =3\}$ of state $3$?

How would I also write down the distribution function $F(x) = \mathbb{P}(Z\le x)$ and the density function $f(x)$ on $x>0$?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

Let $Z_i$ denote the first hitting time of $3$ when $Y_0=i$. Then the distribution of $Z$ is the barycenter of the distributions of $Z_1$ and $Z_2$ with weights $\alpha_1$ and $\alpha_2$.

Furthermore, $Z_1=\frac13T+BZ'_2$ where $(T,B,Z'_2)$ are independent, $T$ is standard exponential, $B$ is Bernoulli with probability $\frac13$ of success, and $Z'_2$ is distributed like $Z_2$.

Likewise, $Z_2=\frac12T'+B'Z'_1$ where $(T',B',Z'_1)$ are independent, $T'$ is standard exponential, $B'$ is Bernoulli with probability $\frac12$ of success, and $Z'_1$ is distributed like $Z_1$.

Let $u_i(s)=\mathrm E(\mathrm e^{-sZ_i})$ and $t(s)=\mathrm E(\mathrm e^{-sT})=(s+1)^{-1}$. Then, $$ u_1(s)=t(\tfrac13s)\cdot(\tfrac13u_2(s)+\tfrac23),\qquad u_2(s)=t(\tfrac12s)\cdot(\tfrac12u_1(s)+\tfrac12), $$ that is, $$ u_1(s)=\frac{2+u_2(s)}{3+s},\qquad u_2(s)=\frac{1+u_1(s)}{2+s}, $$ which yields $$ u_1(s)=\frac{5+2s}{5+5s+s^2},\qquad u_2(s)=\frac{5+s}{5+5s+s^2}. $$ Equivalently, $$ u_1(s)=\frac{a}{1+as}+\frac{1-a}{1+(1-a)s},\qquad u_2(s)=\frac{b}{1+as}+\frac{1-b}{1+(1-a)s}, $$ for $a=\frac12(1+\frac1{\sqrt5})$ and some $b$ in $(0,1)$. Finally, $$ \mathrm E(\mathrm e^{-sZ})=\alpha_1u_1(s)+\alpha_2u_2(s)=\frac{A}{1+as}+\frac{1-A}{1+(1-a)s}, $$ with $A=\alpha_1a+\alpha_2b$, hence the density of the distribution of $Z$ is $$ f(z)=\left(\frac{A}a\mathrm e^{-z/a}+\frac{1-A}{1-a}\mathrm e^{-z/(1-a)}\right)\cdot[z\geqslant0]. $$

share|improve this answer
    
Thanks @Didier! Have been looking through what you did slowly, in order to understand it. Most appreciated again. Cheers :) –  Richard Apr 18 '12 at 23:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.