Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Let the following transition matrix represent a $4$ state Markov chain $$\begin{pmatrix} 0 & a & 0 & b \\ \frac{1}{2} & 0 & \frac{1}{3}+c & d \\ 0 & a & 0 & b \\ e & 0 & f & 0 \end{pmatrix}$$ Let all the constants be positive real numbers, what values of these would make the chain aperiodic? Any help here is greatly appreciated, because I can't see how to do this other than finding an expression for the $n$th power of the matrix at the diagonal values

share|improve this question
    
For this to be the transition matrix of a Markov chain, never mind whether it's aperiodic or not, the rows must sum to $1$. –  Robert Israel Feb 22 '12 at 21:04

2 Answers 2

up vote 2 down vote accepted

If all of $a,b,c,d,e,f$ are $>0$, then the chain is aperiodic. Starting in state 2 you can go $2\to1\to2$ or $2\to4\to1\to2$ with non-zero probability. Hence, the period of state 2 divides both 2 and 3, and so the period of state 2 is 1. Since the chain is irreducible, all states are aperiodic.

If, for example, $a=0$ then the chain is not even irreducible.

share|improve this answer
    
Right, I think I understand, is the only condition on the constants the one you stated? I find this concept of aperiodic hard to get my head around –  Freeman Feb 22 '12 at 23:40

More generally: aperiodicity of a Markov chain depends only on the underlying directed graph. The actual values of the nonzero entries of the transition matrix don't matter, as long as they are nonzero.

EDIT: A picture may help. Here is the directed graph with the states $1,2,3,4$ and an arrow from $i$ to $j$ wherever the $(i,j)$ entry in the transition matrix is nonzero.

enter image description here

Consider starting at a state $i$, travelling along the arrows, and ending back at $i$. If every way of doing this has a number of steps divisible by $d$, and $d$ is the largest integer with this property, then state $i$ has period $d$. If $d=1$, the state is aperiodic. As Byron said, starting, say, at state $2$ you can return to $2$ in $2$ steps ($2 \to 1 \to 2$) or $3$ steps ($2 \to 4 \to 1 \to 2$), so state $2$ is aperiodic. On the other hand, if you removed the arrow from $2$ to $4$ (i.e. made $d=0$), all states would have period $2$.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.