0
$\begingroup$

Consider a Markov's chain on the state space $\{0,1,\ldots\}$ with the transition matrix which is an identity matrix. This gives us that every state communicates only with itself. From the definition of irreducible Markov chain, in an irreducible Markov chain every state must communicate with each other. So in this case the Markov chain is not irreducible.

However, the answer given is "this chain is irreducible and positive recurrent." I don't understand how? Can someone make me understand, please?

$\endgroup$
0
$\begingroup$

Yes, since there there does not exist a path connecting from one node to another node that is there exists two states between which a path is not there, so it is not irreducible as you figured out.

A recurrent state is also known as a Persistent state.

A few notations before proceeding -

$f_{jk}^{(m)} =$ Probability that the given system reaches state $k$ for the first time after $m$ transitions.

A state $j$ is said to be persistent or a recurrent state if $F_{jj} = \sum_{t=1}^{\infty} f_{jj}^{(n)} = 1$ that is starting from state $j$ and returning to state $j$ is certain.

If $F_{jj}<1$, then the state $j$ is called Transient state.

And you see here in your case, $f_{jj}^{(0)} = 1 \forall j = 0,1,2,..,n$

So all the states are recurrent states.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.