1
$\begingroup$

Let a Markov chain with State space $E=\{1,2,3,4\}$ and probability transition matrix:

$$P=\begin{bmatrix} 0 & 1 & 0 & 0 \\ 1/4 & 0 & 1/4 & 1/2\\0 & 1& 0 & 0 \\ 0 &1/2&0&1/2 \end{bmatrix}$$

How can I find the period of the chain?


I think the states 1,2,3 are periodic with period 2 and 4 is aperiodic because there is a loop.

$\endgroup$
11
  • 1
    $\begingroup$ What have you tried? Have you drawn a diagram of the chain? Which states do you think are periodic and which of them are aperiodic? $\endgroup$
    – saz
    May 24, 2015 at 14:53
  • 1
    $\begingroup$ This is not a transition matrix. $\endgroup$
    – Did
    May 24, 2015 at 14:59
  • 1
    $\begingroup$ @GiulyB The chain is aperiodic if all states are aperiodic and so far we do not know this, right? Okay, so there is a loop at state $4$ and therefore $4$ is aperiodic, I agree. What about the remaining ones? Any idea how to prove that they have period 2? $\endgroup$
    – saz
    May 24, 2015 at 15:31
  • 1
    $\begingroup$ @GiulyB Well, what does this tell us? A state $i$ is (in general) not periodic with period $2$ if $p^2(i,i)>0$. What do you really have to show? $\endgroup$
    – saz
    May 24, 2015 at 15:57
  • 1
    $\begingroup$ What you have shown (in your last comment) is that $2 \in N_x$ for all $x \in \{1,2,3,4\}$. This implies $d_x = gcd(N_x) \in \{1,2\}$. It remains to decide whether $d_x = 1$ or $d_x = 2$ for $x \in \{1,2,3\}$. $\endgroup$
    – saz
    May 24, 2015 at 16:29

1 Answer 1

2
$\begingroup$

Set $\DeclareMathOperator \gcd{gcd}$

$$N_x := \{n \in \mathbb{N}; p^n(x,x)>0\} \qquad \quad d_x := \gcd(N_x).$$

  • state 4: Since $p(4,4) = \frac{1}{2}>0$, it follows that $p^n(4,4)>0$ for all $n \in \mathbb{N}$; hence, $N_4 = \mathbb{N}$ and $d_4 = \gcd(\mathbb{N})=1$, i.e. state $4$ is aperiodic.
  • state 3: We have $$p^2(3,3) = \mathbb{P}^3(X_2 = 3) \geq \mathbb{P}^3(X_2 = 3, X_1 = 2) = 1 \cdot \frac{1}{4} >0,$$ i.e. $2 \in N_3$. Moreover, $$\begin{align*} p^5(3,3) &= \mathbb{P}^3(X_5 = 3) \\ &\geq \mathbb{P}^3(X_1 = 2, X_2 = 4, X_3 = 4, X_4 = 2, X_5 = 3) \\ &= 1 \cdot \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{4} >0. \end{align*}$$ Consequently, $5 \in N_3$ and this implies $d_3 \leq \gcd(2,5)= 1$.

The argumentation for state 1 and 2 is similar, I leave it to you.

$\endgroup$
4
  • $\begingroup$ so which is the period of the chain? $\endgroup$
    – GiulyB
    May 24, 2015 at 16:42
  • 1
    $\begingroup$ @GiulyB What conclusion did you draw for state 1 and 2? I won't solve your homework for you. $\endgroup$
    – saz
    May 24, 2015 at 16:45
  • $\begingroup$ also $d_2\leq gdc(2,5)$ and $d_1\leq gdc(2,5)$.All the states have period 1 so the chain is aperiodic. $\endgroup$
    – GiulyB
    May 24, 2015 at 17:02
  • $\begingroup$ @GiulyB Yes, that's correct. $\endgroup$
    – saz
    May 24, 2015 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.