# Are these transient or recurrent states in a Markov chain?

I have the following transition matrix for a Markov chain with states $A, B, C, D, E$:

$$\left| \begin{array}{ccc} 0 & 0 & \frac{1}{2} & \frac{1}{2} & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 \\ \frac{1}{2} & 0 & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & 0 & \frac{1}{2} \\ 0 & 0 & 0 & 0 & 1 \end{array} \right|$$

State $E$ is an absorbing state, but I am wondering how to classify the other states. After drawing out the transition diagram, it seems that all the other states are transient states as eventually we will end up in state $E$. There are no states where we 'get stuck' alternating back and forward. Is this correct?

• According to the definition, a state $i$ is said to be transient, when there is a positive probability that we will never return to state $i$, starting from $i$. That is true in your case, for the reason you mentioned. – thanasissdr Dec 17 '14 at 23:04

• {E} which is closed and hence, state E is a recurrent state and
• {A,B,C,D}, which is a non-closed communicating class. In finite state Markov chains, states belonging to a non-closed communicating classes become transient.
I seem to recall a theorem, in which if an absorbing state is accessible from a communicating class $C$, then all states in $C$ are transient.
Throwing MATLAB at the problem, I find that all but one eigenvalue has magnitude less than 1, and this one eigenvalue has left eigenvector corresponding to the distribution which is only at $E$, so my analysis agrees with yours. Note that there is a pair of complex eigenvalues, which represent cycles (for example the cycle between $A$ and $C$), but because their magnitude is less than 1 these cycles die out in time.