Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

i would like to understand the following statement about Markov chains on a finite state space S:

"If S is finite, then one state ist persistent and all persistent states are non-null."

It is more or less clear to me, that there must exist at least one persistent state. So my question reduces to: Why are all those persisten states non-null ?

Thank you for any help

share|improve this question
add comment

1 Answer 1

up vote 2 down vote accepted

The finiteness of the state space allows us to derive the finiteness of the expected hitting time from very rough bounds. A persistent state $s$ can be reached from any state. Since there are finitely many states, there is a maximal number $n$ of steps that need to be taken from any other state to reach $s$, and a minimal probability $p$ of reaching $s$ in $n$ steps from any other state. Then we can regard the Markov chain as an infinite sequence of attempts to reach $s$. Each attempt takes $n$ steps and has a chance of at least $p$ to hit $s$, so the probability of reaching $s$ after at most $kn$ steps is bounded from below by a geometric distribution, which has a finite expectation value.

share|improve this answer
    
This is very helpful ! –  readingframe Apr 27 '12 at 11:44
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.