1
$\begingroup$

I have the following transition matrix for a Markov chain with states $A, B, C, D, E$:

$$ \left| \begin{array}{ccc} 0 & 0 & \frac{1}{2} & \frac{1}{2} & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 \\ \frac{1}{2} & 0 & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & 0 & \frac{1}{2} \\ 0 & 0 & 0 & 0 & 1 \end{array} \right| $$

State $E$ is an absorbing state, but I am wondering how to classify the other states. After drawing out the transition diagram, it seems that all the other states are transient states as eventually we will end up in state $E$. There are no states where we 'get stuck' alternating back and forward. Is this correct?

$\endgroup$
  • $\begingroup$ According to the definition, a state $i$ is said to be transient, when there is a positive probability that we will never return to state $i$, starting from $i$. That is true in your case, for the reason you mentioned. $\endgroup$ – thanasissdr Dec 17 '14 at 23:04
1
$\begingroup$

You are correct. The following is the state transition diagram associated with the given transition probability matrix. From the diagram, we observe that, there are two communicating classes:

  • {E} which is closed and hence, state E is a recurrent state and

  • {A,B,C,D}, which is a non-closed communicating class. In finite state Markov chains, states belonging to a non-closed communicating classes become transient.

enter image description here

$\endgroup$
0
$\begingroup$

I seem to recall a theorem, in which if an absorbing state is accessible from a communicating class $C$, then all states in $C$ are transient.

$\endgroup$
0
$\begingroup$

Throwing MATLAB at the problem, I find that all but one eigenvalue has magnitude less than 1, and this one eigenvalue has left eigenvector corresponding to the distribution which is only at $E$, so my analysis agrees with yours. Note that there is a pair of complex eigenvalues, which represent cycles (for example the cycle between $A$ and $C$), but because their magnitude is less than 1 these cycles die out in time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.