Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Why we can calculate the limiting probability of $\begin{pmatrix}0.6&0.2&0.1&0.1\\0.6&0&0.3&0.1\\0&0.6&0&0.4\\0&0&0.6&0.4\end{pmatrix}$ in this way?

Solution: Taking $\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}(I-P)=$$\begin{pmatrix}\pi_0&\pi_1&\pi_2&\pi_3\end{pmatrix}\begin{pmatrix}-0.4&0.2&0.1&0.1\\0.6&-1&0.3&0.1\\0&0.6&-1&0.4\\0&0&0.6&-0.6\end{pmatrix}=0$ and we get the limiting probability

I don't quite get why it can be calculated in this way as state 2 is obviously not aperiodic so can this method being used? If not, how should one calculate it?

share|cite|improve this question
State 2 is obviously aperiodic (and every state is obviously aperiodic). – Did Dec 18 '12 at 11:15
aperiodic doesn't mean that at the first step one can return, only that for some N, return at step $n \ge N$ has positive probability. – coffeemath Dec 18 '12 at 11:23
One of you could write that as an answer. – joriki Dec 18 '12 at 14:08
up vote 2 down vote accepted

Call your matrix $M$. Note that both $M^3$ and $M^4$ have all strictly positive entries. This means there is a positive probability that from state $k=1,2,3,4$ that there will be return to state $k$ in three steps, and in four steps. Any integer $t$ greater than $3 \cdot 4-3-4=5$ has an expression $t=3u+4v$ where $u,v \ge 0$. It follows that for $N=6$ we can say that, for any $n \ge N$, the recurrence probability from state $k$ to state $k$ in $n$ steps has positive probability. This shows all four states $1,2,3,4$ are aperiodic, since aperiodic only means positive recurrence probability at $n$ steps, for all sufficiently large values of $n$.

NOTE: For this matrix $M$ in fact all one needs is that each of $M^2$ and $M^3$ has all strictly positive entries on the diagonal. Then for all $n \ge 2$ one can as above show there is positive probability, starting at state $k$, of returning to state $k$ in $n$ steps.

share|cite|improve this answer
"One positive entry on the diagonal + irreducibility" guarantees aperiodicity. – Did Dec 21 '12 at 7:09

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.