1
$\begingroup$

I am trying to determine the period of each state $ j = 0, 1, 2$ for this irreducible Markov Chain with transition probability matrix $$P=\begin{bmatrix}0&0&1\\1&0&0\\\frac{1}{2}&\frac{1}{2}&0\end{bmatrix}$$

All states are of period $1$ because $0 \rightarrow 2 \rightarrow 0$ and $0 \rightarrow 2 \rightarrow 1 \rightarrow 0$ have $gcd(2,3) =1$

Is this correct and is the proof sound?

$\endgroup$
  • 1
    $\begingroup$ Yes, the Markov Chain is aperiodic because of the two cycles you have identified whose lengths are relatively prime. $\endgroup$ – Fabio Somenzi Dec 8 '18 at 6:35
0
$\begingroup$

Comment: Here are some computations in R that I have sometimes found convenient for use with ergodic Markov Chains having small finite state spaces. [If the matrix $\mathbf{P}$ is based on empirical data, make sure each row of the transition matrix sums exactly to $0.]$

Establish ergodicity. $\mathbf{P}^8$ is a power of the transition matrix in the current Question that has all positive elements, so the chain is ergodic.

P = matrix(c(0, 0, 1,
             1, 0, 0,
            .5,.5, 0), nrow=3, byrow=T)
P
     [,1] [,2] [,3]
[1,]  0.0  0.0    1
[2,]  1.0  0.0    0
[3,]  0.5  0.5    0

P2 = P %*% P;  P4 = P2 %*% P2;  P8 = P4 %*% P4;  P8
       [,1]   [,2]   [,3]
[1,] 0.4375 0.1875 0.3750
[2,] 0.3750 0.2500 0.3750
[3,] 0.3750 0.1875 0.4375

Compute limiting distribution. The stationary vector $\sigma$ with $\sigma\mathbf{P} = \sigma$ is also the limiting distribution of the ergodic chain. For an ergodic matrix, the left eigenvector with the largest modulus is real and is proportional to the steady state vector.

eigen(t(P))$vectors     # transpose to get LEFT eigenvectors
              [,1]                  [,2]                  [,3]
[1,] -0.6666667+0i -0.3535534+0.3535534i -0.3535534-0.3535534i
[2,] -0.3333333+0i -0.3535534-0.3535534i -0.3535534+0.3535534i
[3,] -0.6666667+0i  0.7071068+0.0000000i  0.7071068+0.0000000i

s = as.numeric(eigen(t(P))$vectors[,1]);  s = s/sum(s);  s
[1] 0.4 0.2 0.4         # first-listed vector has largest modulus

s %*% P                 # to verify stationarity
     [,1] [,2] [,3]
[1,]  0.4  0.2  0.4

So $\sigma = (.4, .2, .4)$ is the limiting distribution.

Perhaps there is more-elegant R code for this. If so, suggestions are welcome.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.