1
$\begingroup$

I'm a bit confused by this theorem, not sure of the name in english but here are the details:

Let the markov chain $X = (X_n)_{n\geq 0}$ with transition matrix $\mathbb{P}$ and state space $S$.

Let $X$ be irreducible then, if a state in $S$ is transient, then all states are transient. And every state will only be visited a finite number of times.

I'm not quite sure how a markov chain could satisfy this property. I can see how a subset of states might be transient, but to have a markov chain you need at least one recurrent state, perhaps I'm misunderstanding something.

If such a markov chain exists, then I'd like to see an example of one, I've tried searching but I can't seem to find anything on this.

Any help would be greatly appreciated.

|cite|improve this question
$\endgroup$
  • 3
    $\begingroup$ A Markov chain may have infinitely many states, for example the classical random walk on $\Bbb Z^3$ has every sate transient. $\endgroup$ – Lord Shark the Unknown Jun 29 '17 at 6:16
2
$\begingroup$

The assumptions that $X$ is irreducible and that there exists a transient state $z$ imply, that $S$ is not finite. The point is the following: Assume there is a recurrent state $x$. Then there is a $m$ such that you reach $z$ in $m$ steps from $x$ with probability $p>0$. So each time you return to $x$, you throw a $p$-coin if you will reach $z$ in $m$ steps.
As an example, onsider the random walk on $\mathbb{Z}$ which goes to the right with probability $0.99$ and left with probability $0.01$. It should be clear, that the chain goes to $\infty$ then.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ It's not clear to me how you would prove that the states of this irreducible chain are transient. Can you explain further? $\endgroup$ – Lorenzo Najt Jun 29 '17 at 6:26
  • $\begingroup$ I want to use Borel-Cantelli 2. There is a positive probability, that you reach $z$ from $x$ in $m$ steps. If $x$ is recurrent, there is infinitely often the possibility to do so, and because of the Markov property, indepentently. By BC 2 this will occure a.s. infinitely often, a contradiction to the assumption that $z$ is transient. $\endgroup$ – crankk Jun 29 '17 at 6:49
  • $\begingroup$ I mean your example on the integers. $\endgroup$ – Lorenzo Najt Jun 29 '17 at 7:44
  • $\begingroup$ This is the strong law of large numbers. You can write your chain as $X_n=Z_1+...+Z_n$, where $Z_i$ are i.i.d with $P(Z_i=1)=0.99,P(Z_i=-1)=0.01$. By the LLL $X_n/n \to E[Z_i]=0.98$, so $ \lim X_n = \lim 0.98n = \infty$. $\endgroup$ – crankk Jun 29 '17 at 8:12
  • $\begingroup$ It's not clear how that proves that the chain is not recurrent. Even if I drift to infinity, it is plausible that I could visit $0$ infinitely often. $\endgroup$ – Lorenzo Najt Jun 29 '17 at 8:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.