# Given is a Markov chain - classify its states

We have a Markov chain given by the transition matrix

$$M= \begin{pmatrix} 1/3 & 2/3 & 0 & 0 \\ 1/2 & 1/2 & 0 & 0 \\ 1/4 & 0 & 1/4 & 1/2\\ 0 & 0 & 0 & 1 \end{pmatrix}$$

Classify the states of the Markov chain which is given by the transition matrix $M$.

Can you please tell me if I did it correctly? I'm pretty sure this will be asked in the test I write next week!

For a better illustration, I converted this matrix to a graph:

We see, these are accessible states: $(a \rightarrow b),(a \rightarrow c),(b \rightarrow a),(d \rightarrow c)$

communicative states: $(a \leftrightarrow b)$ and the other states are only self communicative, so we have classes:

$C_1= \left\{a,b\right\}$

$C_2 = \left\{c\right\}$

$C_3 = \left\{d\right\}$

And we can also say that the markov chain is not irreducible since we have more than one class (three we have).

$C_2=\left\{c\right\}$ so state $c$ is alone in a class and you cannot escape from that state if you entered it once, thus it is a transient state.

Furthermore $C_3=\left\{d\right\}$ is alone in a class and we have that $p_{ii}=1$ and thus it is an absorbing state.

$C_1=\left\{a,b\right\}$ is not transient because you can always escape from one state to another and so they are recurrent states.

• The graph does not seem quite correct. For example, the probabilities associated with all arrows that stem from $a$ do not sum to $1$.
– OnoL
Feb 2, 2018 at 19:25
• @OnoL's Comment appeared while I was typing my Answer. I agree. // During that time the Question was modified, but the diagram still needs to be fixed. // Another Answer has been changed three times since I posted mind, so I won't comment on it (yet). Feb 2, 2018 at 19:30

Your conclusions seem correct, even though the diagram is not. One problem with the diagram is that you show total probability $1/3 + 1/2 + 1/4 > 1$ on arrows leading out of $a$.

$C_1$ is recurrent (persistent) because its states inter-communicate and there is no way out.

$C_3 = \{d\}$ is absorbing, It can be entered from $b$, but there is no way out (1 on main diagonal).

$C_2$ is transient, but not emphemeral: it leads to itself, but exit is possible.

Given that $C_1$ is visited, there are methods to determine the long-distribution within it.

Given that state $c$ is visited, there are methods to determine the number of steps until absorption into $C_1$ or $C_3.$

Im not sure how your text book defines a matrix of a Markov chain but its more common that the probability of moving from lets say state $2$ to state $3$ (In your case fron b to c) is the element of the second row and third column. It seems you have done vice versa. And as the sums of the rows of the matrix are $1$ it seems you have done it wrong. But you have understood the concepts otherwise.

This is also the reason why the graph is wrong.