1
$\begingroup$

I'm having trouble understanding how to find a limiting distribution. If I have a Markov Chain whose transition probability matrix is: $$ \mathbf{P} = \matrix{~ & 0 & 1 & 2 & 3 & 4 \\ 0 & q & p & 0 & 0 & 0 \\ 1 & q & 0 & p & 0 & 0 \\ 2 & q & 0 & 0 & p & 0\\ 3 & q & 0 & 0 & 0 & p \\ 4 & 1 & 0 & 0 & 0 & 0 } $$

where p>0, q>0 and p+q=1

How would I go about finding the limiting distribution? Thanks for any and all help!

$\endgroup$
1
$\begingroup$

There is no stable limiting distribution. The transition matrix is not diagonalizable.


The Markov chain has this state diagram:

enter image description here

Now suppose that at an arbitrarily large number of steps later, the chain is in state $0$ with probability $a$, state $1$ with probability $b$, state $2$ with probability $c$, state $3$ with probability $d$, and state $4$ with probability $e$, and suppose, for contradiction, that this is a constant limiting distribution.

Then

$$a = bp + e$$

$$b = aq$$

$$c = bq$$

$$d = cq$$

$$e = dq$$

Substitution gives $e = aq^4$, so

$$a = aqp + aq^4 \Rightarrow 1 = qp + q^4$$

We also know $p+q=1$, so

$$1 = q(1-q)+q^4$$

$$q^4 -q^2 + q - 1$$

$$q^2(q+1)(q-1)+1(q-1)=0$$

Since $q \neq 1$,

$$q^3 +q^2 +1 = 0$$

This has no positive solutions for $q$ (by inspection). Therefore, there is no stable limiting distribution.

$\endgroup$
0
$\begingroup$

HINT: Diagonalize the matrix $\mathbf{P}$.

$\endgroup$
  • $\begingroup$ It's not diagonalizable. $\endgroup$ – Zubin Mukerjee Mar 3 '15 at 11:09
0
$\begingroup$

P is a right transition matrix and represents the following Markov Chain:

State Transition Diagram

This finite Markov Chain is irreducible (one communicating class) and aperiodic (there is a self-transition). Thus, it has a limiting distribution which is the solution of

$$ π = π P $$

This limiting distribution corresponds to the normalized left eigenvector of P with eigenvalue 1 and positive entries which is

$$ π = \frac{p^5-1}{p^5-p^4} \begin{bmatrix} \frac{1}{p^4} & \frac{1}{p^3} & \frac{1}{p^2} & \frac{1}{p^1} & 1 \end{bmatrix} $$

Relevant Mathematica notebook here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.