I am working on this problem as part of my first course on Stochastic Processes:

Let $(X_{n})_{n \in \mathbb{N}}$ a Markov Chain taking values on $\{0,1,2,3,...\}$ with transition probabilities $p_{i,i+1} = \alpha_i$ and $p_{i,0} = 1 - \alpha_{i}$, for every $i \geq 0$. Show that

{$0$ is a recurrent state} $\iff \prod\limits_{i=0}^{\infty}\alpha_i = 0 \iff \sum\limits_{i = 0}^{\infty}(1 - \alpha_i) = +\infty$

I believe that I was able to prove the first equivalence. Let $\rho_{i,j} = \mathbb{P}(T_{j} < + \infty|X_{0} = i)$, where $T_j$ is the number of periods the chain takes to hit state $j$ (I believe this is standard notation!).

In order to show that $0$ is a recurrent state, I should show that $\rho_{0,0} = 1$. Well, $1 - \rho_{0,0}$ is the probability of the chain never hitting $0$ in finite time after having started at zero. This implies that

$1-\rho_{0,0} = \alpha_{0}\cdot \alpha_1 \cdot \alpha_2... = \prod\limits_{i=0}^{\infty}\alpha_i$,

which proves the first equivalence. Now, I understand that there are two ways to prove the second part. The first is make the direct reasoning of

$\prod\limits_{i=0}^{\infty}\alpha_i = 0 \iff \sum\limits_{i = 0}^{\infty}(1 - \alpha_i) = +\infty$,

which seems to me a Real Analysis problem once we recognize the probabilities as a bounded sequence of real numbers in $[0,1]$. But I couldn't prove this result on my own using sequences and series techniques.

The other way to prove this result is an indirect reasoning in the sense of

{$0$ is a recurrent state} $\iff$ $\sum\limits_{i = 0}^{\infty}(1 - \alpha_i) = +\infty$

This way would use some tricks regarding the definition of recurrent state ($\rho_{0,0} = 1$) and the structure of this chain. Once more I was not able to solve it. Any ideas? Thanks a lot in advance!


Let $a_i =-\ln \alpha _i$. Then $\sum (1-\alpha _i) =\sum (1-e^{-a_i})$. By L'Hopital's Rule $\frac {1-e^{-x}} {x} \to 1$ as $ x\to 0$. Hence $\frac 1 2 \leq \frac {1-e^{-x}} {x} \leq 2$ whenever $x>0$ is sufficiently small. It follows that the series $\sum (1-\alpha _i)$ diverges iff $\sum a_i $ diverges iff $e^{- \sum a_i } =0$ iff $\prod \alpha _i =0$.

  • $\begingroup$ Sorry, I don't see why the bounds imposed on $\frac{1 - e^{-x}}{x}$ imply the above equivalences! $\endgroup$ – Raul Guarini Jul 1 '18 at 15:33
  • $\begingroup$ They give $\frac {a_i} 2 \leq 1-\alpha_i \leq 2a_i$ for $i$ sufficiently large. Also, $e^{-\sum a_i}=\prod e^{-a_i}=\prod \alpha_i$ $\endgroup$ – Kavi Rama Murthy Jul 1 '18 at 23:44
  • $\begingroup$ Well, but how do you know the behavior of the sequence $a_{i}$? $\endgroup$ – Raul Guarini Jul 1 '18 at 23:50
  • $\begingroup$ If either of the two series converges then $\alpha_i \to 1$, equivalently, $a_i \to 0$. So, if you argue by contradiction you will see that we can always assume that $a_i \to 0$. $\endgroup$ – Kavi Rama Murthy Jul 2 '18 at 0:01
  • $\begingroup$ Perhap I shoud add that if $\prod \alpha_i =a >0$ then $\alpha_i=\frac {\alpha_1 \alpha _2 ... \alpha _{n+1}} {\alpha_1 \alpha _2 ... \alpha _{n}} \to \frac a a =1$. $\endgroup$ – Kavi Rama Murthy Jul 2 '18 at 0:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.