# Do discrete-time Markov chain return probabilities have absolutely summable differences?

Fix an aperiodic discrete-time Markov chain $X_0,X_1,\dots$, with $\mathbb P_i$ denoting probabilities according to $X_0=i$. Define the return probability $$p_{ii}^{(n)} = \mathbb P_i[X_n=i].$$

Is it true that $$\sum_{n=0}^\infty |p_{ii}^{(n)}-p_{ii}^{(n+1)}|<\infty\text{?}$$

I am most interested in the null-recurrent case. For transient states there is a positive answer: the return probabilities are themselves summable. I know that for finite state spaces there is exponential convergence to some stationary state, which also implies a positive answer.

• Did you try to compute $p^{(n)}_{ii}$ for the symmetric random walk on $\mathbb Z$ with transitions $i\to i$ and $i\to i\pm1$, for every $i$ in $\mathbb Z$? – Did Sep 13 '17 at 17:37
• @Did: when the self-transition probability is $1/2$ and $i\to i\pm 1$ probabilities each $1/4$, the probabilities are $2^{-2n}\binom{2n}{n}$ which is non-increasing so the differences sum to at most $1$. I don't know about other other self-transition parameters but my guess is they'd behave similarly at least in the long run. – Dap Sep 13 '17 at 17:48
• Right, random walks look like a dead end. Sorry for the noise. – Did Sep 13 '17 at 19:37