Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I need to show that the markov chain that has transition matrix written below is irreducible. \begin{bmatrix} 0.2 & 0.5 & 0.1 & 0.1 & 0.1 \\ 0.2 & 0.5 & 0.3 & 0 & 0 \\ 0.2 & 0 & 0.4 & 0.4 & 0 \\ 0.2 & 0 & 0.2 & 0.4 & 0.2 \\ 0.2 & 0 & 0 & 0.1 & 0.7 \end{bmatrix}

Is it enough for me to say for $n = 1,2,3,4,5$ we have that $\mathbb{P}(X_1 = n | X_0 =1) > 0$ and $\mathbb{P}(X_1 = 1 | X_0 =n) > 0$? Hence its irreducible. Is there any other (easier) way to show irreducibility?

Also, to calculate the stationary distribution, is the fastest/most convenient way to use $\pi P = \pi$?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

You are right about irreducibility.

You usually find the invariant measure using $\pi=\pi P$ and linear algebra. The invariant probability $\pi$ will be unique, since your chain is irreducible.

But your transition matrix is special, so there is a shortcut. The column sums of $P$ are all equal to one. Such a transition matrix is called doubly stochastic and its unique invariant probability measure is uniform, i.e., $\pi=\left({1\over 5},{1\over 5},{1\over 5},{1\over 5},{1\over 5}\right).$

share|improve this answer
    
Thanks @ByronSchmuland ! Didn't realise that trick for doubly stochastic matrices, thanks for that. May I ask, is there any other way to show irreducibility apart from what I did above? –  Richard Apr 27 '12 at 21:16
1  
@Richard Well, it amounts to the same thing, but you could calculate $P^2$ and note that all its entries are strictly positive. –  Byron Schmuland Apr 27 '12 at 21:23
    
Ahh okay, thanks. Just curious to see if there was any other way, but I think I'll stay with what I did. :) –  Richard Apr 27 '12 at 21:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.