# Showing that $0$ is a recurrent state in a Markov Chain

I am working on this problem as part of my first course on Stochastic Processes:

Let $(X_{n})_{n \in \mathbb{N}}$ a Markov Chain taking values on $\{0,1,2,3,...\}$ with transition probabilities $p_{i,i+1} = \alpha_i$ and $p_{i,0} = 1 - \alpha_{i}$, for every $i \geq 0$. Show that

{$0$ is a recurrent state} $\iff \prod\limits_{i=0}^{\infty}\alpha_i = 0 \iff \sum\limits_{i = 0}^{\infty}(1 - \alpha_i) = +\infty$

I believe that I was able to prove the first equivalence. Let $\rho_{i,j} = \mathbb{P}(T_{j} < + \infty|X_{0} = i)$, where $T_j$ is the number of periods the chain takes to hit state $j$ (I believe this is standard notation!).

In order to show that $0$ is a recurrent state, I should show that $\rho_{0,0} = 1$. Well, $1 - \rho_{0,0}$ is the probability of the chain never hitting $0$ in finite time after having started at zero. This implies that

$1-\rho_{0,0} = \alpha_{0}\cdot \alpha_1 \cdot \alpha_2... = \prod\limits_{i=0}^{\infty}\alpha_i$,

which proves the first equivalence. Now, I understand that there are two ways to prove the second part. The first is make the direct reasoning of

$\prod\limits_{i=0}^{\infty}\alpha_i = 0 \iff \sum\limits_{i = 0}^{\infty}(1 - \alpha_i) = +\infty$,

which seems to me a Real Analysis problem once we recognize the probabilities as a bounded sequence of real numbers in $[0,1]$. But I couldn't prove this result on my own using sequences and series techniques.

The other way to prove this result is an indirect reasoning in the sense of

{$0$ is a recurrent state} $\iff$ $\sum\limits_{i = 0}^{\infty}(1 - \alpha_i) = +\infty$

This way would use some tricks regarding the definition of recurrent state ($\rho_{0,0} = 1$) and the structure of this chain. Once more I was not able to solve it. Any ideas? Thanks a lot in advance!

Let $a_i =-\ln \alpha _i$. Then $\sum (1-\alpha _i) =\sum (1-e^{-a_i})$. By L'Hopital's Rule $\frac {1-e^{-x}} {x} \to 1$ as $x\to 0$. Hence $\frac 1 2 \leq \frac {1-e^{-x}} {x} \leq 2$ whenever $x>0$ is sufficiently small. It follows that the series $\sum (1-\alpha _i)$ diverges iff $\sum a_i$ diverges iff $e^{- \sum a_i } =0$ iff $\prod \alpha _i =0$.

• Sorry, I don't see why the bounds imposed on $\frac{1 - e^{-x}}{x}$ imply the above equivalences! – Raul Guarini Jul 1 '18 at 15:33
• They give $\frac {a_i} 2 \leq 1-\alpha_i \leq 2a_i$ for $i$ sufficiently large. Also, $e^{-\sum a_i}=\prod e^{-a_i}=\prod \alpha_i$ – Kavi Rama Murthy Jul 1 '18 at 23:44
• Well, but how do you know the behavior of the sequence $a_{i}$? – Raul Guarini Jul 1 '18 at 23:50
• If either of the two series converges then $\alpha_i \to 1$, equivalently, $a_i \to 0$. So, if you argue by contradiction you will see that we can always assume that $a_i \to 0$. – Kavi Rama Murthy Jul 2 '18 at 0:01
• Perhap I shoud add that if $\prod \alpha_i =a >0$ then $\alpha_i=\frac {\alpha_1 \alpha _2 ... \alpha _{n+1}} {\alpha_1 \alpha _2 ... \alpha _{n}} \to \frac a a =1$. – Kavi Rama Murthy Jul 2 '18 at 0:25