How to prove that $\sum\pi_i = \sum\frac{1}{E_iT_i} = 1$ in an irreducible Markov chain with stationary distribution $\pi$?

In Durrett's book Chapter 5 Theorem 4.6, It said that if p is irreducible and has stationary distribution, then $$\pi_i=\frac{1}{E_iT_i}$$. Where $$T_i$$ is the first time the markov process returns back to state i. $$E_iT_i$$ is the expected number of times needed to return back to state i given the initial state is $$i$$.

Also in the proof of theorem 4.7 said, if p is ireducible and i is positive recurrent, then, $$\pi_j=\frac{\sum_{n=0}^{\infty}P_i(X_n=j, T_i \gt n)}{E_iT_i}$$ is a stationary distribution.

Does that mean $$\frac{\sum_{n=0}^{\infty}P_i(X_n=j, T_i \gt n)}{E_iT_i} = \frac{1}{E_jT_j}$$? How can we prove that?

And also how can we prove that $$\sum\frac{1}{E_jT_j} = 1$$ in an irreducible Markov chain with stationary distribution $$\pi$$?

• What is your definition of “stationary distribution”? I suspect that the definition includes the property that all probabilities sum to 1. Nov 22, 2020 at 19:06
• @Michael What I don't know is why $\sum\frac{1}{E_iT_i}$ is 1? Can we prove it without using the fact that $\sum\pi_i=1$? Nov 22, 2020 at 19:33

What text are you referring to, exactly? Probability: Theory and Examples Fifth Edition by Rick Durrett? The book proves these theorems in detail, so what part of the proof are you having trouble with?

The theorem you are asking about is numbered 5.5.11, not 4.7 - perhaps you have an old edition of the text? The claim that if $$p$$ is irreducible and $$i$$ positive recurrent, then $$\pi_j = \frac{\sum_{n=0}^\infty \mathbb P(X_n=j, T_i > n\mid X_0=i)}{\mathbb E[T_i\mid X_0=i]}$$ defines a stationary distribution is stated as part of theorem 5.5.12.

As for the statement in the title of the question, if $$\pi$$ is a stationary distribution then by definition it must sum to one. The other equality follows from Theorem 5.5.7 in the book, which states:

Let $$i$$ be a recurrent state, and let $$T=\inf\{n\geqslant 1: X_n=i\}$$. Then $$\mu_i(j) = \mathbb E\left[\sum_{n=0}^{T-1} \mathsf 1_{\{X_n=j\}}\right] = \sum_{n=0}^\infty \mathbb P(X_n=j,T>n\mid X_0=i)$$ defines a stationary measure.

In the text, the author uses Fubini's theorem to show the above. The equality $$\pi_i = \frac1{\mathbb E[T_i\mid X_0=i]}$$ follows from summing $$\mu_i$$ over all states and using Fubini's theorem to show that the sum is in fact equal to $$\mathbb E[T_i\mid X_0=i]$$. Finally, the result that an irreducible and positive recurrent Markov chain has a unique stationary measure up to constant multiples is used. This is Theorem 5.5.9 in the text, and the proof requires a bit of finesse. From this it follows that $$\pi_i = \frac{\mu_i(i)}{\mathbb E[T_i\mid X_0=i]}$$, and since by definition $$\mu_i(i)=1$$ , we have $$\pi_i = \frac{1}{\mathbb E[T_i\mid X_0=i]}$$. Since this equality is true for each state, it remains true when summing over all states, and hence $$\sum_i \pi_i = \sum_i \frac{1}{\mathbb E[T_i\mid X_0=i]},$$ as desired.

• Yes, that's exactly what I read in the book. But that's not the answer I'm looking for. My questions are: Does that mean $\frac{\sum_{n=0}^{\infty}P_i(X_n=j, T_i \gt n)}{E_iT_i} = \frac{1}{E_jT_j}$? How can we prove that? And also how can we prove that $\sum\frac{1}{E_jT_j} = 1$ in an irreducible Markov chain with stationary distribution $\pi$? Nov 22, 2020 at 20:39
• Theorem 5.5.12 said, $\pi_j=\frac{\sum_{n=0}^{\infty}P_i(X_n=j, T_i \gt n)}{E_iT_i}$. Theorem 5.5.11 said $\pi_j=\frac{1}{E_jT_j}$, so they should be the same, isnt it? That is $\mu_i(j)E_jT_j = E_iT_i$ If yes, how can we prove it? Nov 22, 2020 at 20:45
• The proofs you are asking for are given in the text. In particular you should closely read the proof to theorem 5.5.7. Nov 22, 2020 at 20:50
• I did read 5.5.7 and 5.5.9 and 5.5.11, 5.5.12. The question is 5.5.11 said $\pi_j=\frac{1}{E_jT_j}$ while 5.5.12 said $\pi_j=\frac{\mu_i(j)}{E_iT_i}$. They are two different expressions. In the meantime, in the proof of theorem 5.5.12, it refers to 5.5.11 and said $\pi_j=\frac{1}{E_jT_j}$. So guess this two expression represents the same value, but it's not that obvious, so wondering how we can prove it? Nov 22, 2020 at 21:00

I've found the answer myself, it's actually quite easy to prove:

In Durret's book, it's been proved that if $$P$$ is irreducible and recurrent, then $$\mu_a$$ is unique up to a constant multiple, and in its prove of this theorem, we found that the constant is $$\nu(a)$$, i.e. $$\nu(x)=\nu(a)\mu_a(x)$$.

That means $$\mu_i(j)\mu_j(k)=\mu_i(k)$$, here $$\mu_i(j)$$ is the constant.

If we sum both sides over $$k$$,

$$\sum_{k}\mu_i(j)\mu_j(k)=\sum_{k}\mu_i(k)$$

We get

$$\mu_i(j)E_j(T_j)=E_i(T_i)$$

We are done. We get

$$\frac{\mu_i(j)}{E_i(T_i)} = \frac{1}{E_j(T_j)}$$

I explained it in my video today: https://youtu.be/lZrzjxQ3YF8