Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Consider the Markov Chain with the matrix $$ \begin{vmatrix} 0 & 0 & 1 & 0 & 0 \\ 0 & {\frac2 3} & 0 & {\frac 1 3 } & 0 \\ 1 & 0 & 0& 0& 0 \\ 0 & {\frac 3 5} & 0 & {\frac2 5} & 0\\ {\frac1 4} & {\frac 1 2} & 0 & 0 & {\frac1 4}\\ \end{vmatrix} $$

i) Classify the states. I assume this is just {$A, B, C, D, E$} or something similar?

ii) Find the stationary distribution of each irreducible, recurrent sub-chain and obtain the mean recurrence time of each state.

Do I use $wP=w$ here? For each separate column perhaps?

So $(x, y, z, q, 1 - x - y - z - q)\times P = (x, y, z, q, 1 - x - y - z - q)$

How do I then go on to find the mean recurrence time of each state?

I'm kind of new to Markov Chains and I'm finding them quite tricky. If someone could explain this to me I would be very grateful as it would help a lot with my upcoming exams! :)

share|improve this question
1  
Draw a picture with the states and the transitions with positive probability between them. You should see right away that there is 1 transient state and 4 recurrent states, divided into 2 irreducible classes. The recurrence times should be apparent as well. –  Did Mar 30 '13 at 14:18
    
@Did okay so {$A, C$} is recurrent, {$B,D$} is recurrent and $E$ is transient. And I find the stationary distribution between $A$-$C$ and $B$-$D$. How do I obtain the mean recurrence of each state? –  Fred Mar 30 '13 at 16:08
    
How do you obtain the expected return time to a given state for a general irreducible Markov chain? –  Did Mar 30 '13 at 18:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.