# Why we can calculate the limiting probability of $\begin{pmatrix}0.6&0.2&0.1&0.1\\0.6&0&0.3&0.1\\0&0.6&0&0.4\\0&0&0.6&0.4\end{pmatrix}$ in this way?

Why we can calculate the limiting probability of $\begin{pmatrix}0.6&0.2&0.1&0.1\\0.6&0&0.3&0.1\\0&0.6&0&0.4\\0&0&0.6&0.4\end{pmatrix}$ in this way?

Solution: Taking $\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}(I-P)=$$\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}\begin{pmatrix}-0.4&0.2&0.1&0.1\\0.6&-1&0.3&0.1\\0&0.6&-1&0.4\\0&0&0.6&-0.6\end{pmatrix}=0$ and we get the limiting probability

I don't quite get why it can be calculated in this way as state 2 is obviously not aperiodic so can this method being used? If not, how should one calculate it?

-
State 2 is obviously aperiodic (and every state is obviously aperiodic). – Did Dec 18 '12 at 11:15
aperiodic doesn't mean that at the first step one can return, only that for some N, return at step $n \ge N$ has positive probability. – coffeemath Dec 18 '12 at 11:23
One of you could write that as an answer. – joriki Dec 18 '12 at 14:08

Call your matrix $M$. Note that both $M^3$ and $M^4$ have all strictly positive entries. This means there is a positive probability that from state $k=1,2,3,4$ that there will be return to state $k$ in three steps, and in four steps. Any integer $t$ greater than $3 \cdot 4-3-4=5$ has an expression $t=3u+4v$ where $u,v \ge 0$. It follows that for $N=6$ we can say that, for any $n \ge N$, the recurrence probability from state $k$ to state $k$ in $n$ steps has positive probability. This shows all four states $1,2,3,4$ are aperiodic, since aperiodic only means positive recurrence probability at $n$ steps, for all sufficiently large values of $n$.
NOTE: For this matrix $M$ in fact all one needs is that each of $M^2$ and $M^3$ has all strictly positive entries on the diagonal. Then for all $n \ge 2$ one can as above show there is positive probability, starting at state $k$, of returning to state $k$ in $n$ steps.