Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

For a discrete time Markov chain with state-space the non-negative integers, for $j>0$,

$$ p_{j,k} = \begin{cases} p/j & \text{for } k = j+1 \\ 1 - 1/j & \text{for }k=j \\ (1-p)/j & \text{for }k=j-1 \end{cases} $$ and $p_{0,1}=p,\ p_{0,0}=1-p$.

Find the ranges of $p$ such that the process is transient, null-recurrent, and positive-recurrent.

Thanks in advance!

Progress: For showing that it is transient, initially I tried summing $p(n)_{0,0}$ (the probability of returning to state 0 in n steps) from n=1 to infinity and finding the range of p for which this sum converged, which would suggest transience, and diverged, which would suggest recurrence. This however involved a double summation of factorials which was too hard to solve.

Then I tried finding the general solution to $f_{j,0}$, the probability that, starting in j, the process will ever reach state 0. I became stuck at a messy recursion equation: $jf_{j,0}=pf_{j+1,0}+(1-p)f_{j-1,0}+j-1$. Finding p for which $f_{0,0}=1$ would imply recurrence.

Then I tried solving the stationary distribution, pi=pi*P, but this too resulted in a nasty recursion equation: $pi_{j}=p*j/(j-1)*pi_{j-1}+(1-p)*j/(j+1)*pi_{j+1}$. My lecturer says that there is a simplification which can make these recursion equations solvable but I am yet to find see it...

Update: I think I've got it.

share|improve this question
    
I've edited this to put the mathematical notation in $\TeX$. I wonder what was intended where it says $p_{0,1}=p$ and $p_{0,1}=1-p$. Should one of the subscripts have said $0,1$ and the other $1,0$? –  Michael Hardy Sep 2 '11 at 11:59
    
Thanks! Yes, the one should have subscript 0,0, not 0,1. –  James Sep 2 '11 at 12:06
3  
Sounds as homework. Please tag as such, if so; tell us your progress, if you understand the concepts, etc. –  leonbloy Sep 2 '11 at 16:49
    
Like @leon said. Depending on what the OP knows, the appropriate answer can go from one line to slightly more than that. –  Did Sep 2 '11 at 17:58
    
Thanks for the pointers! I'm kind of new to this site so any tips are welcome! Also, I have a feeling that the 'one line answer' is transience: p>0.5, positive-recurrence: p<0.5 and null-recurrence: p=0.5. I will need to show the 'process' of getting there though. –  James Sep 3 '11 at 1:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.