0
$\begingroup$

The transition matrix T = \begin{bmatrix} 3/4 &1/4 \\ 1 &0 \end{bmatrix}

is clearly a regular transition matrix but the chain itself is not aperiodic (although it is irreducible), right?

(My reasoning behind its non-aperiodicity is that while state 1 is aperiodic, state 2 is not since it has a period of 2.)

In a standard linear algebra text, I read that a Markov Process has a unique stationary distribution iff the transition matrix is regular. (The above transition matrix being regular indeed has a unique stationary distribution [4/5 1/5].)

However, I read in a Cover's & Thomas' Elements of Information Theory that: if the Markov Process is aperiodic and irreducible, then it has a stationary distribution that is unique. However, as I have deduced above the matrix T is clearly not aperiodic.

These two ways of describing a finite-state transition matrix are, therefore, quite confusing to me. Any help would be greatly appreciated! Many Thanks!

$\endgroup$
  • 1
    $\begingroup$ State $2$ is also aperiodic. Consider that $\mathbb P(X_2=2\mid X_0=2)$ and $\mathbb P(X_3=2\mid X_0=2)$ are both positive. $\endgroup$ – Math1000 May 2 at 23:11
  • $\begingroup$ Oh yes! you are absolutely right. I think I got messed up in my understanding of aperiodicity of a chain. But, one last question: Could you please clarify if a Markov Process which has a regular transition matrix is also aperiodic and irreducible always? $\endgroup$ – TAH May 2 at 23:25
0
$\begingroup$

A Markov chain with a regular transition matrix is always irreducible and aperiodic.

Let $M$ denote the transition matrix. So there exists $n$ such that $M^n$ has all positive entries. Then clearly it is possible to get from any state to any other state, so the chain is irreducible. Furthermore, this implies $P(X_n = i \mid X_0=i)>0$ for all $i$. Now if $M^n$ has all positive entries, then so does $M^{n+1}$, and so we also have $P(X_{n+1}=i \mid X_0=i)>0$ for all $i$. Since $\gcd(n,n+1)=1$, this implies all states are aperiodic.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.