2
$\begingroup$

I need to show that the markov chain that has transition matrix written below is irreducible. \begin{bmatrix} 0.2 & 0.5 & 0.1 & 0.1 & 0.1 \\ 0.2 & 0.5 & 0.3 & 0 & 0 \\ 0.2 & 0 & 0.4 & 0.4 & 0 \\ 0.2 & 0 & 0.2 & 0.4 & 0.2 \\ 0.2 & 0 & 0 & 0.1 & 0.7 \end{bmatrix}

Is it enough for me to say for $n = 1,2,3,4,5$ we have that $\mathbb{P}(X_1 = n | X_0 =1) > 0$ and $\mathbb{P}(X_1 = 1 | X_0 =n) > 0$? Hence its irreducible. Is there any other (easier) way to show irreducibility?

Also, to calculate the stationary distribution, is the fastest/most convenient way to use $\pi P = \pi$?

$\endgroup$
3
$\begingroup$

You are right about irreducibility.

You usually find the invariant measure using $\pi=\pi P$ and linear algebra. The invariant probability $\pi$ will be unique, since your chain is irreducible.

But your transition matrix is special, so there is a shortcut. The column sums of $P$ are all equal to one. Such a transition matrix is called doubly stochastic and its unique invariant probability measure is uniform, i.e., $\pi=\left({1\over 5},{1\over 5},{1\over 5},{1\over 5},{1\over 5}\right).$

$\endgroup$
  • $\begingroup$ Thanks @ByronSchmuland ! Didn't realise that trick for doubly stochastic matrices, thanks for that. May I ask, is there any other way to show irreducibility apart from what I did above? $\endgroup$ – Richard Apr 27 '12 at 21:16
  • 1
    $\begingroup$ @Richard Well, it amounts to the same thing, but you could calculate $P^2$ and note that all its entries are strictly positive. $\endgroup$ – user940 Apr 27 '12 at 21:23
  • $\begingroup$ Ahh okay, thanks. Just curious to see if there was any other way, but I think I'll stay with what I did. :) $\endgroup$ – Richard Apr 27 '12 at 21:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.