Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is the following statement true:

In a finite Markov chain, if $i$ is a transient state then there is at least one recurrent state $j$ such that $j$ is reachable from $i$.

share|cite|improve this question

I'm not sure if your looking for an algebraic proof, but intuitively I was thinking:

Assume your finite state space is $S = \{1, 2, ... , n\}$. Let $X_i$ denote the state you are in for all $i \in S$.The basic form of a transient "map" would be when

$$X_1 \rightarrow X_2 \rightarrow X_3 \rightarrow ... \rightarrow X_i $$

It is possible that $i = n$. Now when you reach state $i$, you have two options. Either move to another state, or remain in state $i$. There is no option of "nothing happening" in state $i$ as this would just be that you remain in state $i$, i.e the probability of going from $i \rightarrow i = 1$.

Looking at option $1$, if you go from $X_i$ to some other state $X_j$, you have just created a recurrance class between states $X_i$ and $X_j$ and so you have gone from a transient state (say $X_1$) to a recurrant state ($X_i$ or $X_j$).

Looking at option 2, if $X_i \rightarrow X_i$, then clearly state $i$ is recurrant and again, you have gone from a transient state to a recurrant state.

So I think your statement is correct.

share|cite|improve this answer

Yes, for any finite state Markov Chain its state space admits the following Doeblin decomposition $$ S = \sum_{i=1}^n H_i+E $$ where sets $H_i$ are absorbing and recurrent, and $E$ is a set of all transient states. In particular it means that $E$ does not have an absorbing subset and for any $x\in E$ there is $1\leq i\leq n$ such that $H_i$ is reachable from $x$. Since $H_i$ is recurrent, each state of it is reachable from $x$.

This answers your question, however if you will be more precise what is your background on Markov Chains, I could provide an answer that may fir you better.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.