0
$\begingroup$

So the problem is: Let $$P = \begin{bmatrix}0.2 &0.8&0&0&0&0&0 \\0.7 & 0.3&0&0&0&0&0\\0&0&0.3&0.5&0.2&0&0\\0&0&0.6&0&0.4&0&0\\0&0&0&0.4&0.6&0&0\\0&0.1&0.1&0.2&0.2&0.3&0.1\\0.1&0.1&0.1&0&0.1&0.2&0.4 \end{bmatrix}$$

Find $\lim_{n\to \infty}P^{n}$.

So I am thinking maybe I should apply the conclusion I got from the question before this one which is, Let since $j$ is recurrent, non-null and aperiodic, then for any $ i∈E$

$$\lim_{n\to \infty}P^{n}(i, j)= F(i,j)(\pi_j)$$ in which $F(i,j) = P_i(T_1<\infty$), where $T_1$ is the first hitting time of $j$.

But I am not exactly sure how to work this problem out. Appreacie any help!

$\endgroup$
1
$\begingroup$

There are three classes of states in this Markov chain:

\begin{align} C_1 &= \{1,2\}\\ C_2 &= \{3,4,5\}\\ C_3 &= \{6,7\}. \end{align}

$C_1$ and $C_2$ are recurrent, while $C_3$ is transient.

Conditioned on $X_0\in C_1$, we find the limiting distribution to be $$\pi_1=\left(\frac7{15},\frac8{15}\right)$$

Conditioned on $X_0\in C_2$, we find the limiting distribution to be $$\pi_2=\left(\frac6{23},\frac7{23},\frac{10}{23}\right).$$

Let $q_{ij}$ be the probability of absorption into state $j$ conditioned on $X_0=i$, for $i=6,7$ and $j=1,2$. Then we have the system of equations \begin{align} q_{61} &= \frac1{10} + \frac3{10}q_{61} +\frac1{10}q_{71}\\ q_{62} &= \frac12 + \frac3{10}q_{62}+ \frac1{10}q_{72}\\ q_{71} &= \frac15 + \frac25 q_{71}+ \frac15 q_{61}\\ q_{72} &= \frac15 + \frac25 q_{72}+ \frac15 q_{62},\\ \end{align} which yields $$q_{61} = \frac15,\ q_{62}=\frac45,\ q_{71}=\frac25,\ q_{72}=\frac35. $$ It follows that \begin{align} \lim_{n\to\infty}P^n&= \begin{bmatrix} \pi_1(1) & \pi_1(2)&0&0&0&0&0\\ \pi_1(1) & \pi_1(2)&0&0&0&0&0\\ 0&0&\pi_2(3) & \pi_2(4) & \pi_2(5) & 0 & 0\\ 0&0&\pi_2(3) & \pi_2(4) & \pi_2(5) & 0 & 0\\ 0&0&\pi_2(3) & \pi_2(4) & \pi_2(5) & 0 & 0\\ q_{61}\pi_1(1) & q_{61}\pi_1(2) & q_{62}\pi_2(3) & q_{62}\pi_2(4) & q_{62}\pi_2(5) & 0 & 0\\ q_{71}\pi_1(1) & q_{71}\pi_1(2) & q_{72}\pi_2(3) & q_{72}\pi_2(4) & q_{72}\pi_2(5) & 0 & 0\\ \end{bmatrix}\\\\ &=\begin{bmatrix} \frac{7}{15} & \frac{8}{15} & 0 & 0 & 0 & 0 & 0 \\ \frac{7}{15} & \frac{8}{15} & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & \frac{6}{23} & \frac{7}{23} & \frac{10}{23} & 0 & 0 \\ 0 & 0 & \frac{6}{23} & \frac{7}{23} & \frac{10}{23} & 0 & 0 \\ 0 & 0 & \frac{6}{23} & \frac{7}{23} & \frac{10}{23} & 0 & 0 \\ \frac{7}{75} & \frac{8}{75} & \frac{24}{115} & \frac{28}{115} & \frac{8}{23} & 0 & 0 \\ \frac{14}{75} & \frac{16}{75} & \frac{18}{115} & \frac{21}{115} & \frac{6}{23} & 0 & 0 \\ \end{bmatrix}. \end{align}

$\endgroup$
  • $\begingroup$ Thanks a lot! One question: In computing the $q_{61},q_{62}, q_{71},$ and $q_{71}$, I don't see how to construct the equations. For example, in computing $q_{61}$, where do the coeffcients $\frac1{10}, \frac3{10}$ and $\frac1{10}$ come from? $\endgroup$ – Kior Oct 12 '16 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.