Classify the states of the Markov chain which is given by the transition matrix

$$M= \begin{pmatrix} 0 & 1/2 & 1/2 & 0 \\ 1/3 & 0 & 0 & 2/3\\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix}$$

It's important for me to fully understand how this is done because I'm pretty sure this will be asked in a test soon :s

I hope you can tell me if I did it correctly and if not how to do it correctly?

I start by converting this matrix to a graph because it's easier for me to classify the states: enter image description here

Accessible states: $(a \rightarrow b),(a \rightarrow c),(b \rightarrow a),(b \rightarrow d),(c \rightarrow a),(d \rightarrow c)$

Communicative states: $(a \leftrightarrow b),(a \leftrightarrow c)$ ,(and thus also $(b \leftrightarrow c)$)

Thus we have $2$ equivalence classes: $C_1=\left\{a,b,c\right\}$ and $C_2=\left\{d\right\}$

Since we have $2$ equivalence classes, the Markov Chain is not irreducible.

Since states $a,b,c$ are in the same equivalence class, e.g. communicate with each other, we can conclude that they are recurrent states.

But state $d$ is also recurrent because it's impossible to not reach this state again from another state.

So there is no state which is transient and there is no state which is absorbable.


We say that state $j$ is accessible from state $i$ , written as $i \to j$, if $p^{(n)}_{ij}>0$ for some $n$. Note that $n$ need not be $1$, $i$ can reach $j$ in a few steps.

In this case, each state is accessible from any states since any states can be reached eventually. All states communicate with each other.

Hence, we only have one equivalence class and it is irreducible and the class is recurrent (i.e. there are no transisent state).

Note that we can reach $d$ from $b$.

No state is absorbing state since no state has an arrow that loop back to itself with probability $1$.

  • $\begingroup$ Thank you very much I read these on several pages because our script is not very detailled and your answer help me understand it complete correct. I write all these example on paper and I think it will be very good for exam! Thank you again!! $\endgroup$ – roblind Feb 4 '18 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.