Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Can anyone give me a simple example of an irreductible (all elements communicate) and transient markov chain? I can't think of any such chain, yet it exists (but has to have an infinite number of elements) thanks

share|improve this question
    
Think about random walks. –  Hans Engler Nov 21 '12 at 23:36
    
But why is it transient? I would need to compute the probability of going back to origin but I don't know how –  lezebulon Nov 21 '12 at 23:45
add comment

1 Answer 1

up vote 4 down vote accepted

A standard example is asymmetric random walk on the integers: consider a Markov chain with state space $\mathbb{Z}$ and transition probability $p(x,x+1)=3/4$, $p(x,x-1)=1/4$. There are a number of ways to see this is transient; one is to note that it can be realized as $X_n = X_0 + \xi_1 + \dots + \xi_n$ where the $\xi_i$ are iid biased coin flips; then the strong law of large numbers says that $X_n/n \to E[\xi_i] = 1/2$ almost surely, so that in particular $X_n \to +\infty$ almost surely. This means it cannot revisit any state infinitely often.

Another example is simple random walk on $\mathbb{Z}^d$ for $d \ge 3$. Proving this is transient is a little more complicated but it should be found in most graduate probability texts.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.