Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

This is exercise 1.7.4 in Norris' Markov Chains textbook. I'm having difficulty calculating a simple looking expectation.

Let $(X_n)_{n\geq0}$ be a simple random walk on $\mathbb{Z}$ with transition probabilities $p_{i,i-1}=q<p=p_{i,i+1}$ where $p+q=1$ and $q>0$. Let $\gamma^0_i=\mathbb{E}_0(\sum_{n=0}^{T_0-1}1_{\{X_n=i\}})$, that is the expected time spent in $i$ between visits to $0$. Find $\gamma^0_i$.

I've tried conditioning on $T_0$ but it led to a sum of probabilities that I found tough to evaluate. I've also tried to analyse it as a random walk on $\mathbb{Z_{\geq0}}$ to hopefully make use of the hitting probabilities but got nowhere. Any hints?

(There is a second part to this question using textbook results that suggest that $\gamma^0_i=(p/q)^i$ for $i \leq0$ and $\gamma^0_i=1$ for $i\geq0$, if I calculated it correctly.)

share|cite|improve this question
Does $\mathbb E_0$ denote the expected value conditioned on return to $0$? – joriki Dec 19 '12 at 11:46
It denotes the expected value conditional on $X_0=0$. – E.Lim Dec 19 '12 at 13:43
So $T_0$ can be infinite? It would be slightly unusual to subtract $1$ from infinity :-) – joriki Dec 19 '12 at 13:51
Yes, it can be infinite. The recurrent case is covered by a theorem in the book, maybe I'll try to adapt the proof to get a better idea what's going on. – E.Lim Dec 19 '12 at 13:55
Ok, a better formulation would be $\gamma^0_i=\mathbb{E_0}(\sum_{n=0}^{\infty}1_{\{X_n=i\text{ and }n+1\leq T_0\}})$ – E.Lim Dec 19 '12 at 14:19

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.