1
$\begingroup$

Starting at some state $i$, we have probability of going $P_{i,i+1} = p$ and probability $P_{i,i-1} = 1-p$ what is the probability I reach N before I reach zero?

Can I convert this to a gambler's ruin problem where I start with i in bank and reach N before I reach zero?

I think the idea is to use the stochastic difference equation: $f_{i+1} - f_i = \frac{q}{p}(f_i - f_{i-1})$, where $f_0 = 0$ and $f_N = 1$ is the absorbing state and this gives rise to $f_i = \frac{1-(q/p)^i}{1-(q/p)^N}$

This gives me the probability of reaching N and not reaching zero, but does it also mean the probability of reaching N before 0? If not, how can I find that?

$\endgroup$
  • 1
    $\begingroup$ "This gives me the probability of reaching N and not reaching zero, but does it also mean the probability of reaching N before 0?" Actually what "this gives" you is "the probability of reaching N before 0". $\endgroup$ – Did Nov 18 '13 at 19:23
  • $\begingroup$ SO does this mean when i am doing a race between two paths in a chain, to compute $k$ finishes before $j$, I set $f_k = 1 and f_j =0 $, can you explain why this makes intuitive sense? $\endgroup$ – knk Nov 18 '13 at 19:37
0
$\begingroup$

Your formula is correct, assuming $q=1-p$; here's argument in biological terms. This Markov chain is a birth-death process and the concept you are looking for is the fixation probability (called the absorption probability in general). Formula (1) on this page gives the probability of absorbing at $N$ starting at state $i$, where $\gamma_k = \frac{1-p}{p}$ for your process. Formula (1) simplifies to your formula by using also the formula for a finite geometric series -- the ones cancel because the sums start at 1.

The fixation probability gives the probability of the process absorbing in the state $N$ when starting from state $i$; one minus this probability gives the fixation probability for state $i=0$ (since there are no other absorbing states). Your second question, the probability of reaching $N$ before $0$, is the same since $0$ and $N$ are both (and the only) absorbing states.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.