1
$\begingroup$

We have a Markov chain given by the transition matrix

$$M= \begin{pmatrix} 1/3 & 2/3 & 0 & 0 \\ 1/2 & 1/2 & 0 & 0 \\ 1/4 & 0 & 1/4 & 1/2\\ 0 & 0 & 0 & 1 \end{pmatrix}$$

Classify the states of the Markov chain which is given by the transition matrix $M$.

Can you please tell me if I did it correctly? I'm pretty sure this will be asked in the test I write next week!

For a better illustration, I converted this matrix to a graph:

enter image description here We see, these are accessible states: $(a \rightarrow b),(a \rightarrow c),(b \rightarrow a),(d \rightarrow c)$

communicative states: $(a \leftrightarrow b)$ and the other states are only self communicative, so we have classes:

$C_1= \left\{a,b\right\}$

$C_2 = \left\{c\right\}$

$C_3 = \left\{d\right\}$

And we can also say that the markov chain is not irreducible since we have more than one class (three we have).

$C_2=\left\{c\right\}$ so state $c$ is alone in a class and you cannot escape from that state if you entered it once, thus it is a transient state.

Furthermore $C_3=\left\{d\right\}$ is alone in a class and we have that $p_{ii}=1$ and thus it is an absorbing state.

$C_1=\left\{a,b\right\}$ is not transient because you can always escape from one state to another and so they are recurrent states.

$\endgroup$
  • 3
    $\begingroup$ The graph does not seem quite correct. For example, the probabilities associated with all arrows that stem from $a$ do not sum to $1$. $\endgroup$ – OnoL Feb 2 '18 at 19:25
  • 1
    $\begingroup$ @OnoL's Comment appeared while I was typing my Answer. I agree. // During that time the Question was modified, but the diagram still needs to be fixed. // Another Answer has been changed three times since I posted mind, so I won't comment on it (yet). $\endgroup$ – BruceET Feb 2 '18 at 19:30
3
$\begingroup$

Your conclusions seem correct, even though the diagram is not. One problem with the diagram is that you show total probability $1/3 + 1/2 + 1/4 > 1$ on arrows leading out of $a$.

$C_1$ is recurrent (persistent) because its states inter-communicate and there is no way out.

$C_3 = \{d\}$ is absorbing, It can be entered from $b$, but there is no way out (1 on main diagonal).

$C_2$ is transient, but not emphemeral: it leads to itself, but exit is possible.

Given that $C_1$ is visited, there are methods to determine the long-distribution within it.

Given that state $c$ is visited, there are methods to determine the number of steps until absorption into $C_1$ or $C_3.$

$\endgroup$
3
$\begingroup$

Im not sure how your text book defines a matrix of a Markov chain but its more common that the probability of moving from lets say state $2$ to state $3$ (In your case fron b to c) is the element of the second row and third column. It seems you have done vice versa. And as the sums of the rows of the matrix are $1$ it seems you have done it wrong. But you have understood the concepts otherwise.

This is also the reason why the graph is wrong.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.