Why we can calculate the limiting probability of $\begin{pmatrix}0.6&0.2&0.1&0.1\\0.6&0&0.3&0.1\\0&0.6&0&0.4\\0&0&0.6&0.4\end{pmatrix}$ in this way?

Solution: Taking $\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}(I-P)=$$\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}\begin{pmatrix}-0.4&0.2&0.1&0.1\\0.6&-1&0.3&0.1\\0&0.6&-1&0.4\\0&0&0.6&-0.6\end{pmatrix}=0$ and we get the limiting probability

I don't quite get why it can be calculated in this way as state 2 is obviously not aperiodic so can this method being used? If not, how should one calculate it?

  • 1
    $\begingroup$ State 2 is obviously aperiodic (and every state is obviously aperiodic). $\endgroup$ – Did Dec 18 '12 at 11:15
  • 2
    $\begingroup$ aperiodic doesn't mean that at the first step one can return, only that for some N, return at step $n \ge N$ has positive probability. $\endgroup$ – coffeemath Dec 18 '12 at 11:23
  • $\begingroup$ One of you could write that as an answer. $\endgroup$ – joriki Dec 18 '12 at 14:08

Call your matrix $M$. Note that both $M^3$ and $M^4$ have all strictly positive entries. This means there is a positive probability that from state $k=1,2,3,4$ that there will be return to state $k$ in three steps, and in four steps. Any integer $t$ greater than $3 \cdot 4-3-4=5$ has an expression $t=3u+4v$ where $u,v \ge 0$. It follows that for $N=6$ we can say that, for any $n \ge N$, the recurrence probability from state $k$ to state $k$ in $n$ steps has positive probability. This shows all four states $1,2,3,4$ are aperiodic, since aperiodic only means positive recurrence probability at $n$ steps, for all sufficiently large values of $n$.

NOTE: For this matrix $M$ in fact all one needs is that each of $M^2$ and $M^3$ has all strictly positive entries on the diagonal. Then for all $n \ge 2$ one can as above show there is positive probability, starting at state $k$, of returning to state $k$ in $n$ steps.

  • $\begingroup$ "One positive entry on the diagonal + irreducibility" guarantees aperiodicity. $\endgroup$ – Did Dec 21 '12 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.