1
$\begingroup$

The transition matrix T = \begin{bmatrix} 3/4 &1/4 \\ 1 &0 \end{bmatrix}

is clearly a regular transition matrix but the chain itself is not aperiodic (although it is irreducible), right?

(My reasoning behind its non-aperiodicity is that while state 1 is aperiodic, state 2 is not since it has a period of 2.)

In a standard linear algebra text, I read that a Markov Process has a unique stationary distribution iff the transition matrix is regular. (The above transition matrix being regular indeed has a unique stationary distribution [4/5 1/5].)

However, I read in a Cover's & Thomas' Elements of Information Theory that: if the Markov Process is aperiodic and irreducible, then it has a stationary distribution that is unique. However, as I have deduced above the matrix T is clearly not aperiodic.

These two ways of describing a finite-state transition matrix are, therefore, quite confusing to me. Any help would be greatly appreciated! Many Thanks!

$\endgroup$
2
  • 1
    $\begingroup$ State $2$ is also aperiodic. Consider that $\mathbb P(X_2=2\mid X_0=2)$ and $\mathbb P(X_3=2\mid X_0=2)$ are both positive. $\endgroup$
    – Math1000
    May 2, 2019 at 23:11
  • $\begingroup$ Oh yes! you are absolutely right. I think I got messed up in my understanding of aperiodicity of a chain. But, one last question: Could you please clarify if a Markov Process which has a regular transition matrix is also aperiodic and irreducible always? $\endgroup$
    – TAH
    May 2, 2019 at 23:25

1 Answer 1

0
$\begingroup$

A Markov chain with a regular transition matrix is always irreducible and aperiodic.

Let $M$ denote the transition matrix. So there exists $n$ such that $M^n$ has all positive entries. Then clearly it is possible to get from any state to any other state, so the chain is irreducible. Furthermore, this implies $P(X_n = i \mid X_0=i)>0$ for all $i$. Now if $M^n$ has all positive entries, then so does $M^{n+1}$, and so we also have $P(X_{n+1}=i \mid X_0=i)>0$ for all $i$. Since $\gcd(n,n+1)=1$, this implies all states are aperiodic.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .