Let a Markov Chain with $|I|=n+1$ states $I=\{0,1,...,n\}$.

For state $i\neq 0$ the probability to move in the next step to $0$ is $p_i\gt 0$, and with probability $1-p_i$ to stay in $i$.

When we are in state $0$ the probability to move in the next step to $i\neq 0$ is $a_i$ such that:

$$\sum_{i\in I\backslash \{0\}} a_i=1$$

Suppose the experiment starts in state $0$, what is the expected value of the number of moves until the next visit to zero?

For convenience let's call this random varible $S$.

What is the stationary distribution?

My approach:

For the expected value of $S$: $$\mathbb{E}[S]=1+\sum_{i\in I\backslash \{0\}} \frac{a_i}{p_i}$$ For the stationary distribution:

I have noticed that if there is some state $i\neq 0$ such that $p_i\lt1$ than the chain is ergodic and otherwise it has a period $2$. But I don't know how to put it all together.


S = no of visits before returning to state 0 Range (S) ={1,2,...}=N

Now,to find E[S]

Apply the lemma E[s] = $\sum_{k=1}^{\inf} P[S\geq k]$

Now, P[$S\geq k$] = p[$\cup_i X_n= i \forall 1 \leq n \leq k-1]$

=$\sum_{i=1}^{n+1} a_i(1-p_i)^{k-1}$(Since,disjoint union)

So, E[S] = $\sum_{k=1}^{\inf}[\sum_{i=1}^{n+1} a_i(1-p_i)^{k-1}]$

=$\sum_{i=1}^{n+1}a_i[\sum_{k=1}^{\inf}(1-p_i)^{k-1}]$(By fubinni 's exchange)

= $\sum_{i=1}^{n+1}[a_i/(p_i)]$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.